tag:blogger.com,1999:blog-85461987320266668622024-03-13T04:37:41.026-07:00Mule's Other EarMule's Earhttp://www.blogger.com/profile/01650109925841837119noreply@blogger.comBlogger23125tag:blogger.com,1999:blog-8546198732026666862.post-72483879504968687102020-10-01T10:24:00.002-07:002020-10-05T09:36:22.958-07:00NodeMCU - Enabling Lua 5.3<p>I could write this as a 1 line post, but I'm going to pad it out a little.</p><p>So, if you've ever used Lua on a NodeMCU device, like an ESP-8266, you know that it defaults to Lua 5.1. Not that there is anything wrong with that, but if you read the <a href="https://nodemcu.readthedocs.io/en/latest/lua53/" target="_blank">whitepaper</a>, you'll see that 5.3 has some neat features. But the question I couldn't find the answer to is: How do you cause the project to compile with Lua 5.3.</p><p><span></span></p><a name='more'></a>I'm as good as the next guy when it comes to web searches, maybe better than average. My success rate is pretty high, but I couldn't find the right search terms for that one. AFAIK - this will be the first blog post I've found about it; Which is the reason I'm writing it.<p></p><p>I build my nodeMCU firmware natively, on a Linux box running FC-32, but I'm pretty sure it doesn't matter how you build yours. <br /></p><p>The quick answer:</p><p></p><blockquote><span style="font-family: courier;">$ export LUA=53 && make </span></blockquote><p>I couldn't get it to build in the master branch or the dev branch, but I checked out the release branch and it built just fine there. I flashed it to my D1 Mini Pro and voilĂ .</p><p></p><blockquote><p><span style="font-family: courier;">> print(_VERSION)</span></p><p><span style="font-family: courier;">Lua 5.3 </span></p></blockquote><p>Just like that. I'd rather not explain how I figured it out, makes it seem like I'm smarter than I am. But it involved some poking around the source code until eventually I found, in /app/Makefile. Some lucky 'grep'ing.</p><p></p><blockquote><span style="font-family: courier;"># Handle Lua Directory selector<br />ifeq ("$(LUA)","53")<br /> LUA_DIR := lua53<br />else<br /> LUA_DIR := lua<br />endif</span></blockquote>I hope that helps.<p></p><p>PD</p><p>Note: You may need to <span style="font-family: courier;">make clean</span> if you've been building against Lua 5.1<br /></p><p>Note2: To my surprise, it was still necessary to use '=' before displaying the results of function calls when using the command line. E.g.: <span style="font-family: courier;">wifi.sta.getip()</span> exits silently, whereas <span style="font-family: courier;">=wifi.sta.getip()</span> will display the results of that call. That changed after 5.2 but I guess it didn't make it to the nodeMCU version. Odd.</p><p></p>Mule's Earhttp://www.blogger.com/profile/01650109925841837119noreply@blogger.com0tag:blogger.com,1999:blog-8546198732026666862.post-32076679650250507762019-06-16T09:31:00.000-07:002019-06-21T12:29:08.029-07:00Easy IOT with ESP8266 and MicroPython using sockets instead of HTMLThe problem I've always had with the ESP8266 was that all of the examples showed wrapping the controls in HTML. So - you'll start up a small web server, then decode the GET from the client and run your control logic from there. I don't like that.<br />
<br />
I can see certain advantages, if you <i>want</i> to manually control your outputs from your browser, and you want to actually see the state of your inputs, it's great. Have at it. But I don't want to do that - at least not all of the time. I want an easy way to read the inputs and control the outputs without having to log in with a browser or use some funky CURL.<br />
<br />
<a name='more'></a><br />
<br />
My idea is to have my control application running on a central server, like a Raspberry Pi, and I want to communicate directly with my peripheral (IOT) devices. Most of the logic will happen on the Pi. I want to be able to control one IOT device based on sensors located on a different IOT device.<br />
<br />
So I started researching MicroPython sockets. I've used them before for different projects. It seems so much cleaner. The code much easier to write. The client connects to the socket server and sends a message. The server checks the message and reacts, then sends a reply. Like I said, much easier to implement.<br />
<br />
You can see the initial project on <a href="https://github.com/mule-ear/uPythonIOTSocket" target="_blank">github</a>. It includes a sample client.<br />
<br />
The server programming is as simple as reacting immediately to the response, or defining functions and triggering them from the client message. For example:<br />
<blockquote>
from_client = str(incoming_data)<br />
if from_client == '6on':<br />
led(0)<br />
reply = "Relay On"<br />
elif from_client == '6off':<br />
led(1)<br />
reply = "Relay Off"<br />
elif from_client == "info":<br />
reply = DEVICE<br />
elif from_client == "voltage":<br />
reply = str(get_voltage(adc, 12, 5))</blockquote>
Where DEVICE is a global string with a device description and get_voltage is a function. Check out the github repo.<br />
<br />
This is just the start of the thing. There is still a lot of work to be done. It is not very robust, and breaks pretty easily. It is not secure. But I am very happy with it so far, and I am rarely happy with my own stuff. It is a good start.<br />
<br />
Hopefully I'll get some help along the way and this turns into something really cool. It has a lot of potential.<br />
<br />
Resources:<br />
<a href="https://docs.micropython.org/en/latest/esp8266/tutorial/intro.html#" target="_blank">MicroPython on the ESP8266</a><br />
<a href="https://docs.python.org/3/library/socket.html#example" target="_blank">Python Sockets</a><br />
<a href="https://wiki.wemos.cc/products:d1:d1_mini" target="_blank">Wemos D1 Mini</a><br />
<br />
UPDATE:<br />
I added a sample ruby client, in case that's your preferred language.<br />
<br />
UPDATE 2:<br />
Added a C++ client, same reason.<br />
<br />
UPDATE 3:<br />
This works, and it's good as it is, but using it, I realize that the server is blocking, and will only react when the client requests it. It should not be blocking, and be able to perform a normal program - yet still be able to react to a client request. This means threading. Guess I know where to go next :)<br />
<br />
UPDATE 4:<br />Crap - threading isn't included in the Wemos D1 Mini Micropython build. That really would have been too easy. I <i>think</i> I see a way around it. Still digging.Mule's Earhttp://www.blogger.com/profile/01650109925841837119noreply@blogger.com0tag:blogger.com,1999:blog-8546198732026666862.post-49390398474935971242017-10-27T13:56:00.001-07:002017-10-28T11:58:14.080-07:00Monitoring My Pi Cluster WIth Nagios, Part III - Checking the other Pi's using check_by_ssh<br />
It's been too long since I wrote my other Nagios posts. The server I had running died, and I didn't recover the nagios files from it. Fortunately, I had an AWS machine to compare to, so I didn't lose everything.<br />
<br />
Setting up the check_by_ssh checks is, like all of nagios (it seems), a multi-config process. You'll need to modify <span style="font-family: "georgia" , "times new roman" , serif;">$NAGIOS_HOME/etc/objects/commands.cfg, $NAGIOS_HOME/etc/nagios.cfg, and $NAGIOS_HOME/etc/servers/<server_name>.</span><br />
<span style="font-family: "georgia" , "times new roman" , serif;"><br /></span>
<span style="font-family: "georgia" , "times new roman" , serif;"></span><br />
<a name='more'></a><br />
<br />
<br />
In <a href="http://mule2ear.blogspot.com/2017/08/monitoring-my-pi-cluster-with-nagios_23.html" target="_blank">my last nagios post</a>, I showed how to add servers for nagios monitoring. By uncommenting the line #cfg_dir=/usr/local/nagios/etc/servers - it will cause nagios to read any file in that directory during start-up. This is where I put my pi config files.<br />
<br />
Next, I added remote commands to $NAGIOS_HOME/etc/objects/commands.cfg. Here is an example of how to write those:<br />
<br />
<blockquote class="tr_bq">
# 'check_remote_disk' command definition<br />
define command{<br />
command_name check_remote_disk<br />
command_line $USER1$/check_by_ssh -H $HOSTADDRESS$ -C "$USER1$/check_disk -w $ARG1$ -c $ARG2$ -p $ARG3$"<br />
}<br />
<br />
# 'check_remote_load' command definition<br />
define command{<br />
command_name check_remote_load<br />
command_line $USER1$/check_by_ssh -H $HOSTADDRESS$ -C "$USER1$/check_load -w $ARG1$ -c $ARG2$"<br />
}</blockquote>
<br />
That may not format too well - the command_line is one continuous line.<br />
<br />
Next, add a file for the server you want to check to the $NAGIOS_HOME/etc/servers/ directory. Here is a trimmed down example with the check_by_ssh commands that I added to $NAGIOS_HOME/etc/objects/commands.cfg file(above):<br />
<br />
<blockquote class="tr_bq">
# including the define host section<br />
<br />
define host{<br />
use linux-server ; Name of host template to use<br />
; This host definition will inherit all variables that are defined<br />
; in (or inherited by) the linux-server host template definition.<br />
host_name Raspberry_Pi_1<br />
alias Raspberry_Pi_1<br />
address 192.168.22.101<br />
}<br />
<br />
# including the host group as well<br />
define hostgroup{<br />
hostgroup_name pi-servers ; The name of the hostgroup<br />
alias Raspberry pi servers infrastructure Servers <br />
; Long name of the group<br />
members Raspberry_Pi_1 <br />
; Comma separated list of hosts that belong to this group<br />
<br />
# and the service definitions:<br />
define service{<br />
use local-service ; Name of service template to use<br />
host_name Raspberry_Pi_1<br />
service_description Root Partition<br />
check_command check_remote_disk!20%!10%!/<br />
}<br />
<br />
define service{<br />
use local-service ; Name of service template to use<br />
host_name Raspberry_Pi_1<br />
service_description Current Load<br />
check_command check_remote_load!5.0,4.0,3.0!10.0,6.0,4.0<br />
}</blockquote>
With all of that added, restart nagios and it will show the new raspberry pi that you just added. Hopefully this is enough to get you going. Feel free to copy and modify the other commands in $NAGIOS_HOME/etc/objects/commands.cfg file for broader coverage.<br />
<br />
As always, please let me know if you have any questions, or there is any of this that needs clarification.<br />
<br />
<br />
<br />Mule's Earhttp://www.blogger.com/profile/01650109925841837119noreply@blogger.com0tag:blogger.com,1999:blog-8546198732026666862.post-10016189261286809432017-08-23T08:34:00.001-07:002017-08-23T13:09:00.326-07:00Monitoring My Pi Cluster WIth Nagios, Part II - Adding the Pi'sIn <a href="http://mule2ear.blogspot.com/2017/08/monitoring-my-pi-cluster-with-nagios.html" target="_blank">Part 1</a>, I compiled Nagios from source, installed it and got it running.<br />
<br />
Here, I'm going to describe how I added my Pi's.<br />
<br />
<br />
<a name='more'></a>It's hard toknow where to start first, the config files are inter-related. I guess I'll start where I started, but I'd imagine it will be confusing because it didn't work. So bear with me and I'll try my best to explain as I go.<br />
<br />
<br />
The config files are located in <span style="font-family: "georgia" , "times new roman" , serif;">/usr/local/nagios/etc</span>. If I refer to a config file - it will be in that directory, or a sub-directory. Inside that directory is a sub-directory called <span style="font-family: "georgia" , "times new roman" , serif;">objects</span>. I started out by copying the file <span style="font-family: "georgia" , "times new roman" , serif;">localhost.cfg</span> to <span style="font-family: "georgia" , "times new roman" , serif;">new_server.cfg</span>. I then changed the IP address and host_name properties and tried restarting Nagios.<br />
<br />
Nagios ignored the file completely.<br />
<br />
I had to let Nagios know that the file existed and to use it. I opened <span style="font-family: "georgia" , "times new roman" , serif;">nagios.cfg</span> and copied the line for <span style="font-family: "georgia" , "times new roman" , serif;">localhost.cfg</span> - substituting <span style="font-family: "georgia" , "times new roman" , serif;">new_server.cfg</span> for <span style="font-family: "georgia" , "times new roman" , serif;">localhost.cfg</span>. The line looks like: <span style="font-family: "georgia" , "times new roman" , serif;">cfg_file=/usr/local/nagios/etc/objects/new_server.cfg</span>. <br />
<br />
Nagios did not like that file and would not restart. If I remember correctly, the error was pretty generic - "bad config file" or something similar. The reason was that inside <span style="font-family: "georgia" , "times new roman" , serif;">localhost.cfg</span> it defines a hostgroup. The new server was in the same hostgroup as localhost (as defined in both config files). So, I created a new hostgroup. The other way I could have solved this would have been to add <span style="font-family: "georgia" , "times new roman" , serif;">new_server</span> to the <span style="font-family: "georgia" , "times new roman" , serif;">linux-servers</span> hostgroup in <span style="font-family: "georgia" , "times new roman" , serif;">localhost.cfg</span>. A third way would have been to remove the hostgroup definition from <span style="font-family: "georgia" , "times new roman" , serif;">new_server.cfg</span> completely.<br />
<br />
What you need to take away from that first error is this:<br />
1) The definitions must be unique.<br />
2) All elements must be defined. <br />
<br />
This is from the localhost.cfg file. I changed the host_name
and alias, but notice that the hostgroup is in the same file as the
host, and that the members of the hostgroup has only this host. What I'm
driving at is that they match. <br />
<br />
<blockquote class="tr_bq">
# Define a host for the local machine<br />
<br />
define host{<br />
use linux-server ; Name of host template to use<br />
; This host definition will inherit all variables that are defined<br />
; in (or inherited by) the linux-server host template definition.<br />
host_name Pine64_Nagios_Server<br />
alias Pine64 Nagios Server<br />
address 127.0.0.1<br />
}<br />
<br />
# Define an optional hostgroup for Linux machines<br />
<br />
define hostgroup{<br />
hostgroup_name linux-servers ; The name of the hostgroup<br />
alias Linux Servers ; Long name of the group<br />
members Pine64_Nagios_Server ; Comma separated list of hosts that belong to this group<br />
}<br />
</blockquote>
<br />
So - a little planning can go a long way. If you want to group your servers (nice for display purposes, and easier to wrap your head around), make sure that they are defined in a hostgroup. You can create a separate file for the hostgroup and put them in there (e.g. <span style="font-family: "georgia" , "times new roman" , serif;">pi_servers_hostgroup.cfg</span>).<br />
<br />
Then, by reading the comments in <span style="font-family: "georgia" , "times new roman" , serif;">nagios.cfg</span>, I found a better way - I created a directory called servers and uncommented the line in <span style="font-family: "georgia" , "times new roman" , serif;">nagios.cfg</span> that causes nagios to read all files in that directory: <span style="font-family: "georgia" , "times new roman" , serif;">cfg_dir=/usr/local/nagios/etc/servers</span>.<br />
<br />
I mv'd <span style="font-family: "georgia" , "times new roman" , serif;">new_server.cfg</span> over to the servers directory and restarted nagios. It worked as expected. I then created a server config file for each of the servers that I wanted to monitor. I added all of the servers to the hostgroup - which is now a separate file. Pretty much, I just copied the working server config file, changed the IP address, and, with vim - '<span style="font-family: "georgia" , "times new roman" , serif;">:%s/old_hostname/new_hostname/g</span>' and saved it.<br />
<br />
Now all the servers have been added - but the checks don't work. Ping and ssh work - but all of the others are really only checking the nagios host. Next post - I'll explain ssh_check and how to monitor remote servers.Mule's Earhttp://www.blogger.com/profile/01650109925841837119noreply@blogger.com0tag:blogger.com,1999:blog-8546198732026666862.post-39708959869369419502017-08-18T22:02:00.000-07:002017-08-25T17:20:40.441-07:00Monitoring My Pi Cluster WIth Nagios, Part 1 - InstallingThis will hopefully be Part 1 of the story. BTW - It seems to be pronounced Nah - gee (as in geese) - ose. I installed this on a pine64 to monitor my Pi network (4- Pi3s with MySQL5.7 servers, and 1 Pi2 running Zookeeper) and to learn how to use it.<br />
<br />
I know this is going to be like "somebody who's played racquetball 3 times teaching someone else to play for the first time", but here goes:<br />
<br />
<a href="https://www.rifftrax.com/ator-the-fighting-eagle" target="_blank">HT RiffTrax Ator the Fighting Eagle</a> <br />
<br />
<a name='more'></a><br />
<br />
Installing Nagios was pretty easy - they have a really good instruction page <a href="https://support.nagios.com/kb/article/nagios-core-installing-nagios-core-from-source.html#Debian" target="_blank">here</a>. It boils down to:<br />
<blockquote class="tr_bq">
apt-get update<br />
apt-get upgrade<br />
apt-get install -y autoconf gcc libc6 make wget unzip apache2 apache2-utils php5 libgd2-xpm-dev<br />
cd /tmp<br />
wget -O nagioscore.tar.gz https://github.com/NagiosEnterprises/nagioscore/archive/nagios-4.3.2.tar.gz<br />
tar xzf nagioscore.tar.gz<br />
cd nagioscore-nagios-4.3.2/<br />
./configure --with-httpd-conf=/etc/apache2/sites-enabled<br />
make all<br />
useradd nagios<br />
usermod -a -G nagios www-data<br />
make install<br />
make install-init<br />
systemctl enable nagios.service<br />
make install-commandmode<br />
make install-config<br />
make install-webconf<br />
a2enmod rewrite<br />
a2enmod cgi<br />
htpasswd -c /usr/local/nagios/etc/htpasswd.users nagiosadmin<br />
systemctl restart apache2.service<br />
systemctl start nagios.service<br />
apt-get install -y autoconf gcc libc6 libmcrypt-dev make libssl-dev wget bc gawk dc build-essential snmp libnet-snmp-perl gettext<br />
<b>chmod u+s /bin/ping</b><br />
cd /tmp<br />
wget --no-check-certificate -O nagios-plugins.tar.gz https://github.com/nagios-plugins/nagios-plugins/archive/release-2.2.1.tar.gz<br />
tar zxf nagios-plugins.tar.gz<br />
cd /tmp/nagios-plugins-release-2.2.1/<br />
./tools/setup<br />
./configure<br />
make<br />
make install<br />
systemctl restart nagios.service</blockquote>
That installs Nagios and the basic plug-ins. Please <b>NOTE </b>that I slipped in <span style="font-family: "verdana" , sans-serif;">chmod u+s /bin/ping</span>. Debian allows only root to ping, but nagios needs it too.<br />
<br />
When you do the htpasswd bit - you'll be required to enter the password for your Nagios administrator. You might want to write that down somewhere.<br />
<br />
You'll also need an MTA if you want to receive emails from Nagios - I used postfix. I don't know this guy, but I did check out his <a href="http://tombuntu.com/index.php/2009/12/22/send-outgoing-email-with-postfix/" target="_blank">blog post about installing postfix</a>. That will save me from having to describe it here.<br />
<br />
The commands for sending mail won't work out-of-the-box with postfix, but they won't work with sendmail either. I'm not sure what package provides /bin/mail, but I didn't spend much time looking for it. If dpkg doesn't know, I'm going to use something else.<br />
<br />
So, visit this guys blog to see how he altered his commands.cfg file to use postfix. It works. Dang it, I forgot to copy the link for that blog post - so I won't be able to give credit to him.<br />
<blockquote class="tr_bq">
# 'notify-host-by-email' command definition<br />
define command{<br />
command_name notify-host-by-email<br />
command_line /usr/bin/printf "%b" "Subject: $NOTIFICATIONTYPE$ Host Alert: $HOSTNAME$ is $HOSTSTATE$\n\n***** Nagios *****\n\nNotification Type: $NOTIFICATIONTYPE$\nHost: $HOSTNAME$\nState: $HOSTSTATE$\nAddress: $HOSTADDRESS$\nInfo: $HOSTdOUTPUT$\n\nDate/Time: $LONGDATETIME$\n" | /usr/sbin/sendmail -vt $CONTACTEMAIL$<br />
}<br />
<br />
# 'notify-service-by-email' command definition<br />
define command{<br />
command_name notify-service-by-email<br />
command_line /usr/bin/printf "%b" "Subject: $NOTIFICATIONTYPE$ Service Alert: $HOSTALIAS$/$SERVICEDESC$ is $SERVICESTATE$\n\n***** Nagios *****\n\nNotification Type: $NOTIFICATIONTYPE$\n\nService: $SERVICEDESC$\nHost: $HOSTALIAS$\nAddress: $HOSTADDRESS$\nState: $SERVICESTATE$\n\nDate/Time: $LONGDATETIME$\n\nAdditional Info:\n\n$SERVICEOUTPUT$\n" | /usr/sbin/sendmail -vt $CONTACTEMAIL$<br />
}</blockquote>
Nagios support has another way, and I'll try that one on an AWS server. It should be noted that the contacts.cfg file needs to be altered - putting your email address in there. Also - check out templates.cfg to make sure you're using the right host template - unless you want 24x7 notification.<br />
<br />
So - right now you should be able to log-in, and Nagios is monitoring itself. In part two, I'll talk about adding servers and monitoring them. I'll also cover a little bit about how Nagios works.<br />
<br />
And - to start Nagios - `sudo /usr/local/bin/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg` Mule's Earhttp://www.blogger.com/profile/01650109925841837119noreply@blogger.com0tag:blogger.com,1999:blog-8546198732026666862.post-56202042341200859652017-08-13T18:01:00.000-07:002017-08-21T18:18:28.856-07:00Pinebook: Installing ArmbianI just got my Pinebook last week. If you don't know what a Pinebook is: <a href="https://www.pine64.org/?page_id=3707" target="_blank">here</a>. I have a couple of pine64's and they are pretty cool. I'm using them as servers, and not using the GPIO or Euler Bus. But the Pinebook is different.<br />
<br />
Disclaimer: If you brick your Pinebook, I'm sorry - this is just how I installed Armbian on mine. <br />
<br />
<a name='more'></a>Please see Update 3 at the bottom - It was much easier to use Armbian's install tool. My way worked - but there was a lot more effort involved.<br />
<br />
The first thing I had to do was, of course, charge the battery. I guess I've learned patience over the years.<br />
<br />
After charging the battery, and turning it on, I was able to get a feel for the machine. I have to say that it is one slick little machine. I was only able to get the 14" version, 11" was not an option. And I waited quite a while for it (them). I had originally registered on the waiting list with one email address - but I waited so long that I figured it got lost or something - so I registered with another email address. This gave me the opportunity to purchase two, which I did. The second is on its way.<br />
<br />
Aside from the extravagant shipping costs, it's a really inexpensive machine (US $99). It doesn't have a LAN port, but that's about all it's lacking. WiFi, Bluetooth, Micro-SD slot, and two USB ports. I don't know if either is USB-3, tbh.<br />
<br />
The distro it shipped with has issues, though. During my initial drive - I caught a segmentation fault from Java. This is easy enough to work around - I <a href="http://www.webupd8.org/2012/09/install-oracle-java-8-in-ubuntu-via-ppa.html" target="_blank">installed Oracle's Java</a> which worked fine. But then I ran '<span style="font-family: "courier new" , "courier" , monospace;">man</span>' and it was broke. I've been using Linux for a while now, and why memorize stuff when you have man right there to remind you? I have a low threshold for that kind of thing. I didn't want to invest a bunch of time and find out something else was broke.<br />
<br />
I have had such good luck with <a href="https://www.armbian.com/" target="_blank">Armbian</a> that I wanted to give it a try. They only have <a href="https://www.armbian.com/pinebook-a64/" target="_blank">Ubuntu versions available for Pinebook</a>, which is OK. I prefer Debian, but the Pinebook version comes with a root user built-in. I'm so happy with Armbian that I donated a small amount (More than a beer, but less than a Pinebook).<br />
<br />
I actually got the image using Pine's <span style="font-family: "courier new" , "courier" , monospace;">pine64-installer</span>, which I got from their website. I downloaded the available Armbian image and burned it to a SD card, and booted the Pinebook with it installed. When an SD card is inserted, the Pinebook defaults to booting the SD card instead of the emmc. This gave me a chance to try it out and see if anything was broken. Everything worked fine, including man.<br />
<br />
Then, I shut down the Pinebook and removed the SD card. I copied the image (from <span style="font-family: "courier new" , "courier" , monospace;">~/.config/pine64-installer/downloadedImage</span>) to the card, and restarted the Pinebook. By that, I mean I cp'd the image to <span style="font-family: "courier new" , "courier" , monospace;">/home/panchod/</span> on the SD card. The short version is that I then dd'd the image from the SD card to <span style="font-family: "courier new" , "courier" , monospace;">/dev/mmcblk0</span>. (<span style="font-family: "courier new" , "courier" , monospace;">sudo dd if=/home/panchod/<imageFileName> of=/dev/mmcblk0 bs=4M</span>) Rebooted (without the SD card inserted) and boom - a working Armbian computer.<br />
<br />
I say short version because I tried some stuff in the middle there. Once I realized (thanks to parted) that /dev/mmcblkboot0 was not a part of mmcblk0, I tried writing the image directly to the root of the volume. This is what worked.<br />
<br />
I haven't tried it yet - but there's a shortcut on the default XFCE4 desktop called "Config". Opening that yields an ncurses application - and one of the options of that application is <Install>. I bet that would work, too. I'll let you know when the new one gets here. I ordered the 64GB emmc, so I'll have to change that out first.<br />
<br />
All-in-all I'm very pleased with the Pinebook so far. Just for a goof, I compiled sqlite on it, and it works great. I have MySQL5.7 compiled for a pine64 - I wonder if it will work on the Pinebook. Armbian came with <span style="font-family: "courier new" , "courier" , monospace;">build-essential</span> already built-in. I'll scp the client over - don't really need a server when I've got one running on the pine64.<br />
<br />
That's it - please let me know if you run into any issues - or if you have an easier way.<br />
<br />
Oh, yeah - almost forgot. I don't want to complain about pine64.org, but I'm going to anyway. I didn't try to reach tech support, pretty sure it only exists as a wiki and a forum. I didn't find anything on their forum about writing an 'outside' distro to the emmc. They only have two images that will automatically write themselves to the emmc: One is Android and the other is Ubuntu/Mate. I didn't want the Ubuntu/Mate one - I'm thinking that's the same as what the computer came with.<br />
<br />
Linux is about freedom - and I think it would be a good idea to be make it easy to change distros on the Pinebook. I've been dying to give <a href="http://www.linuxfromscratch.org/" target="_blank">Linux From Scratch</a> another go - and it would be awesome if I could do it on a Pinebook<br />
<br />
Update 1: Both USB ports are 2.0<br />
<br />
Update 2: The MySQL5.7 client that I compiled for the Pine64 works just fine on Pinebook. I thought it would. I just now checked apt-cache and it says they have mysql-server5.7 available (and default). I guess I didn't need to compile it after all - but I would swear that it wasn't available at that time.<br />
<br />
Update 3: My new Pinebook came today, and I replaced the emmc module with the 64GB one (which had an Android image on it already). I ran Armbian off of the Micro-SD card, and this time I installed to the emmc using the Config program I mentioned above. It worked flawlessly.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiDEg8_GiRW0gDksKeZd1lYaE6duRGkKOGUyQ_ihCwB8aLhE7Z9RzgTpjIkt4Yzy-_5EslxKC2BF2Msbz2iAs9r_sbrPrRMZRnws-QNLrkik_NZwuQwu2T1OpEKJ7UWFSbgRDuaYj_NG8w/s1600/Screenshot_2017-08-21_19-16-21.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="401" data-original-width="588" height="218" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiDEg8_GiRW0gDksKeZd1lYaE6duRGkKOGUyQ_ihCwB8aLhE7Z9RzgTpjIkt4Yzy-_5EslxKC2BF2Msbz2iAs9r_sbrPrRMZRnws-QNLrkik_NZwuQwu2T1OpEKJ7UWFSbgRDuaYj_NG8w/s320/Screenshot_2017-08-21_19-16-21.png" width="320" /></a></div>
Mule's Earhttp://www.blogger.com/profile/01650109925841837119noreply@blogger.com2tag:blogger.com,1999:blog-8546198732026666862.post-85682823780663731072017-03-09T14:18:00.000-08:002017-06-20T10:35:27.645-07:00Using dhclient to automatically change IP addresses of Raspberry Pi on start-upSo, in <a href="http://mule2ear.blogspot.com/2017/03/one-way-to-deal-with-changing-ip.html" target="_blank">my previous post</a> I dealt with changing IP addresses on my Pi cluster by scp'ing the new IP addresses after the Pi had started up and established a network, and creating an ssh/config file from that info.<br />
<br />
This is fine for most occasions. But I have one instance set-up to run zookeeper. I'd have to go around and change the config files on all my other Pi's. Ansible could cure this, and I'll get there one of these days. But I'm not there today. And actually - I think I'll use the script I wrote previously to update ansibles' hosts file. I just now thought of that.<br />
<br />
Anyway - I want the zookeeper Pi to always have the same address.<br />
<br />
<a name='more'></a><br /><br />
I combined <a href="https://superuser.com/questions/487607/how-to-request-a-specific-ip-address-from-dhcp-server" target="_blank">this question</a> on <a href="http://superuser.com/">superuser.com</a> with what I learned about creating systemd start-up jobs to request the same ip address from my hotspot.<br />
<br />
The gist of the superuser.com answer is add <span style="font-family: "courier new" , "courier" , monospace;">send dhcp-request-address 192.168.0.XXX</span> to <span style="font-family: "courier new" , "courier" , monospace;">/etc/dhcp/dhclient.conf</span>.<br />
<br />
I then created a short executable script in <span style="font-family: "courier new" , "courier" , monospace;">/usr/bin</span> called <span style="font-family: "courier new" , "courier" , monospace;">request_ip.sh</span>.<br />
<blockquote class="tr_bq">
$cat /usr/bin/request_ip.sh <br />
#! /bin/bash<br />
<br />
dhclient -r -v && dhclient -4 -d -v -cf /etc/dhcp/dhclient.conf wlan0</blockquote>
Then I created the service file:<br />
<blockquote class="tr_bq">
# cat /etc/systemd/system/request_ip.service <br />
[Unit]<br />
Description=Change ip to 131 for myProject <br />
After=network-online.target<br />
<br />
[Service]<br />
Type=oneshot<br />
ExecStart=/usr/bin/request_ip.sh<br />
<br />
[Install]<br />
WantedBy=multi-user.target<br />
WantedBy=graphical.target</blockquote>
I installed request_ip.service in <span style="font-family: "courier new" , "courier" , monospace;">/etc/systemd/system</span>, and enabled it with <span style="font-family: "courier new" , "courier" , monospace;">systemctl enable request_ip.service</span>. Checking syslog, I know it ran. I'll update this post if I run into problems.Mule's Earhttp://www.blogger.com/profile/01650109925841837119noreply@blogger.com0tag:blogger.com,1999:blog-8546198732026666862.post-33271414618401751502017-03-09T13:07:00.000-08:002017-06-20T21:02:55.988-07:00One way to deal with changing IP addresses on my raspberry Pi clusterOne of the annoyances I have with my router (it's actually a hotspot) is that every time I start up my Pi cluster, they all have different IP addresses than they had the day before.<br />
<br />
So - I have two ways to deal with this.<br />
<br />
The first way - I scp the IP addresses as a file (after the pi's start up) to my work computer, then generate an ssh_config file. The second way, I request the same IP from my hotspot. All my Pi's are running Debian Jessie with systemd.<br />
<br />
<h2>
The first way: </h2>
scp (or you could email it) the IP address after networking has started up.<br />
<br />
<a name='more'></a><br />
<br />
<h3>
The links:</h3>
<a href="https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/">Running Services After the Network is up</a><br />
<a href="https://bbs.archlinux.org/viewtopic.php?id=166618" target="_blank">[Solved] SystemD, how to start a service after network is up.</a><br />
<br />
<h3>
Overview:</h3>
To do this with systemd, I had to create the script to scp the ip, make sure it was executable and then copied to <span style="font-family: "courier new" , "courier" , monospace;">/usr/bin/</span>.<br />
<br />
Then, I had to create a systemd service file, copy that to <span style="font-family: "courier new" , "courier" , monospace;">/etc/systemd/system/</span> and enable the service.<br />
<br />
<h3>
The Details:</h3>
<h4>
The script to scp the IP address:</h4>
<blockquote class="tr_bq">
$cat send_ip_address.sh <br />
#! /bin/bash<br />
<br />
IP_ADDR=`ifconfig wlan0 | grep 'inet addr:' | cut -f 12 -d ' ' | cut -c 6-`<br />
echo $IP_ADDR > /tmp/$HOSTNAME<br />
<br />
scp -i /home/panchod/.ssh/id_rsa -o StrictHostKeyChecking=no /tmp/$HOSTNAME panchod@192.168.0.174:/home/panchod/lan/ip_addresses/</blockquote>
You'll undoubtedly notice <span style="font-family: "courier new" , "courier" , monospace;">-o StrictHostKeyChecking=no</span>. This is to avoid the nastiness of the host not being in the known_hosts file.<br />
<br />
<h4>
The service file:</h4>
<br />
<blockquote class="tr_bq">
$ cat send_ip.service <br />
[Unit]<br />
Description=Send ip address to my Toshiba on startup<br />
After=network-online.target<br />
<br />
[Service]<br />
Type=oneshot<br />
ExecStart=/usr/bin/send_ip_address.sh<br />
<br />
[Install]<br />
WantedBy=multi-user.target<br />
WantedBy=graphical.target</blockquote>
And then I wrote a script to install everything:<br />
<blockquote class="tr_bq">
$ cat set_up.sh <br />
#! /bin/bash<br />
<br />
sudo cp -iv send_ip_address.sh /usr/bin<br />
sudo cp -iv send_ip.service /etc/systemd/system/send_ip.service<br />
sudo systemctl enable send_ip.service</blockquote>
<br />
I made <span style="font-family: "courier new" , "courier" , monospace;">set_up.sh</span> executable. Put them all in the same directory. Tar'd and zip'd the directory. Then, just scp'd the tarball to each pi, untar'd the tarball and ran <span style="font-family: "courier new" , "courier" , monospace;">set_up.sh</span>. Done and done. I should have used ansible - but I'm not there yet.<br />
<br />
On my work machine - it's kind of kooky how I have it set up, so it's probably not what you'd do. Because I use ssh for work, and therefore can't mess with <span style="font-family: "courier new" , "courier" , monospace;">.ssh/config</span> (because it's shared) I'll tell you what I do instead.<br />
<br />
I have an alias set up for my local lan and use a separate config file.<br />
<blockquote class="tr_bq">
alias lssh='ssh -F ~/lan/ssh_config'</blockquote>
<span style="font-family: "courier new" , "courier" , monospace;">ssh_config</span> looks like this:<br />
<blockquote class="tr_bq">
$ cat ~/lan/ssh_config <br />
Host localCluster.*<br />
User root<br />
Port 22<br />
IdentityFile /home/panchod/.ssh/id_rsa-new_pi_net<br />
<br />
<br />
Host localCluster.bpi-iot-ros-ai<br />
HostName 192.168.0.198<br />
Host localCluster.raspberry-pi-1<br />
HostName 192.168.0.144<br />
Host localCluster.raspberry-pi-2<br />
HostName 192.168.0.154<br />
Host localCluster.raspberry-pi-3<br />
HostName 192.168.0.153<br />
Host localCluster.raspberry-pi-4<br />
HostName 192.168.0.117<br />
Host localCluster.raspberry-pi-6<br />
HostName 192.168.0.131</blockquote>
and I have bash completion set-up for it:<br />
<blockquote class="tr_bq">
#=====================================================================<br />
# LAN SSH Autocomplete<br />
#=====================================================================<br />
<br />
_complete_lssh_hosts()<br />
{<br />
COMPREPLY=()<br />
cur="${COMP_WORDS[COMP_CWORD]}"<br />
comp_lssh_hosts=`cat ~/lan/ssh_config| \<br />
grep "^Host " | \<br />
awk '{print $2}' | grep -v "\*"<br />
`<br />
COMPREPLY=( $(compgen -W "${comp_lssh_hosts}" -- $cur) )<br />
return 0<br />
}<br />
complete -F _complete_lssh_hosts lssh</blockquote>
So - when I type <span style="font-family: "courier new" , "courier" , monospace;">lssh <tab></span> it starts off with <span style="font-family: "courier new" , "courier" , monospace;">localCluster.</span><span style="font-family: inherit;"> I can then hit <span style="font-family: "courier new" , "courier" , monospace;"><r><tab></span> for raspberry-pi and it fills in the rest, then just pick which one I want.</span><br />
<br />
<span style="font-family: inherit;">I also had to make a script to generate the <span style="font-family: "courier new" , "courier" , monospace;">ssh_config</span> file. This is a little more complicated than it needs to be, but there was a reason I did it this way.</span><br />
<br />
<span style="font-family: inherit;">First - I have all the pi stuff in <span style="font-family: "courier new" , "courier" , monospace;">~/lan/</span>. As you can see above - the IP addresses are in <span style="font-family: "courier new" , "courier" , monospace;">~/lan/ip_addresses</span>. I put the header of<span style="font-family: "courier new" , "courier" , monospace;"> </span><span style="font-family: "courier new" , "courier" , monospace;">ssh_config</span> in <span style="font-family: "courier new" , "courier" , monospace;">~/lan</span> and called it <span style="font-family: "courier new" , "courier" , monospace;">ssh_config_preface.txt</span>. It looks like this:</span><br />
<blockquote class="tr_bq">
<span style="font-family: inherit;"> Host localCluster.*<br /> User root<br /> Port 22<br /> IdentityFile /home/panchod/.ssh/id_rsa-new_pi_net</span></blockquote>
<br />
The script to generate the config that you see above:<br />
$cat set_up_ssh.sh<br />
<blockquote class="tr_bq">
#! /bin/bash<br />
<br />
BASE_DIR=/home/panchod/lan<br />
SSH_FILE=$BASE_DIR/ssh_config<br />
CLUSTER_FILE=$BASE_DIR/cluster<br />
<br />
cat /home/panchod/lan/ssh_config_preface.txt > $SSH_FILE<br />
<br />
if grep localCluster $CLUSTER_FILE ;<br />
then<br />
echo "Cluster exists in $CLUSTER_FILE - replacing it"<br />
sed -i 's/localCluster.*/localCluster /g' $CLUSTER_FILE<br />
else<br />
echo "Cluster doesn't exists creating it"<br />
echo "localCluster " >> $CLUSTER_FILE<br />
fi<br />
<br />
for i in /home/panchod/lan/ip_addresses/* ; do<br />
<br />
NAME=`basename $i`<br />
#echo $NAME<br />
IP=`cat $i`<br />
#echo $IP<br />
<br />
echo "Host localCluster.$NAME<br />
HostName $IP" >> $SSH_FILE<br />
sed -i "s/^localCluster.*$/& $IP/g" $CLUSTER_FILE<br />
done<br />
<br />
echo $SSH_FILE<br />
cat $SSH_FILE<br />
echo; echo<br />
echo $CLUSTER_FILE<br />
cat $CLUSTER_FILE </blockquote>
<br />
I also use cssh, so I have an alias for that as well: <span style="font-family: "courier new" , "courier" , monospace;">alias lcssh='cssh -c /home/panchod/lan/cluster'</span>.<br />
<br />
I should add a test to ensure that the files exist, but this isn't production, and I only wrote them for me, so if they fail - no one to blame but myself.<br />
<br />
[Edit: changed the location of .ssh_config for consistency] Mule's Earhttp://www.blogger.com/profile/01650109925841837119noreply@blogger.com0tag:blogger.com,1999:blog-8546198732026666862.post-73995052045798919622017-01-28T12:50:00.003-08:002017-06-20T10:38:02.090-07:00HOW-TO use Kafka to stream Postgresql traffic with bottledwater_pg on a Raspberry Pi 3If you want to stream changes to your Postgresql database to Kafka on your Raspberry Pi 3 - this is how I did it. I'm sure there are other ways to accomplish this. All I want to do here is to show you how I managed it. I'll preface my comments with a hash '#'.<br />
<br />
<a name='more'></a><br /><br />
First: The links.<br />
<a href="https://github.com/confluentinc/bottledwater-pg" target="_blank">bottledwater_pg</a><br />
<a href="https://kafka.apache.org/" target="_blank">Apache Kafka</a><br />
<a href="https://zookeeper.apache.org/" target="_blank">Apache Zookeeper</a><br />
<a href="https://www.postgresql.org/" target="_blank">Postgresql</a><br />
<a href="https://www.confluent.io/" target="_blank">Confluent</a><br />
<a href="https://avro.apache.org/" target="_blank">Apache Avro</a> <br />
<br />
Here goes:<br />
<blockquote class="tr_bq">
sudo apt-get update && sudo apt-get -y upgrade<br />
sudo apt-get install oracle-java8-jdk git</blockquote>
I'm pretty sure you only need a jre, but I installed the jdk for a different project<br />
<br />
<a href="http://mule2ear.blogspot.com/2017/01/install-zookeeper-package.html" target="_blank">Install zookeeper</a> and start<br />
This is optional - the confluent package has a limited version of zookeeper<br />
<br />
<a href="http://mule2ear.blogspot.com/2017/01/install-postgresql-from-source.html" target="_blank">Compile and install postgresql from source</a><br />
The version of postgresql provided by Debian jessie is older than that required by bottledwater_pg<br />
<br />
Adjust the values of pg_hba.conf and postgresql.conf. I used 'kafka_user' but that's up to you:<br />
<blockquote class="tr_bq">
sudo -i<br />
echo "wal_level = logical<br />
max_wal_senders = 8<br />
wal_keep_segments = 4<br />
max_replication_slots = 4" >> /usr/local/pgsql/data/pg_hba.conf<br />
<br />
<br />
echo "local replication kafka_user trust<br />
host replication kafka_user 127.0.0.1/32 trust<br />
host replication kafka_user ::1/128 trust" >> /usr/local/pgsql/data/postgresql.conf</blockquote>
<br />
You're going to add that user "kafka_user" to postgres. This assumes that you followed the link above and created the database 'kafka_user':<br />
<blockquote class="tr_bq">
export PATH=$PATH:/usr/local/pgsql/bin<br />
psql -Upostgres -c "create role kafka_user with SUPERUSER LOGIN;"<br />
createdb -Upostgres kafka_user<br />
psql -Upostgres -c "REVOKE CONNECT ON DATABASE kafka_user FROM PUBLIC;"<br />
psql -Upostgres -c "GRANT CONNECT ON DATABASE kafka_user TO kafka_user;"</blockquote>
Now you need <a href="http://mule2ear.blogspot.com/2017/01/installing-c-bindings-for-avro.html" target="_blank">Apache Avro</a>, or at least the C and maybe C++ portions. I couldn't get the complete avro package to compile out-of-the-box. Originally I installed all of the pre-requisites, but ended up just installing the C and C++ portions.<br />
<br />
Get the confluent package:<br />
<br />
<blockquote class="tr_bq">
wget http://packages.confluent.io/archive/3.0/confluent-3.0.1-2.11.zip<br />
sudo unzip confluent-3.0.1-2.11.zip -d /opt</blockquote>
Install bottledwate_pg:<br />
<blockquote class="tr_bq">
sudo -i<br />
cd /opt</blockquote>
<blockquote>
git clone https://github.com/confluentinc/bottledwater-pg.git<br />
apt-get install libsnappy-dev librdkafka-dev libcurl4-openssl-dev libpq-dev<br />
export PKG_CONFIG_PATH=/lib/pkgconfig/<br />
ln -s /usr/local/pgsql/bin/pg_config /usr/bin/pg_config<br />
make<br />
make install</blockquote>
I added the kafka_user to my Raspberry Pi because I didn't want to figure out the right postgres connection string:<br />
<blockquote class="tr_bq">
sudo useradd kafka_user</blockquote>
Almost there. Now just follow the instructions on bottledwater_pg's github page:<br />
<blockquote class="tr_bq">
# start kafka server as root (zookeeper should already be running)<br />
cd /opt/confluent<br />
./bin/kafka-server-start ./etc/kafka/server.properties<br />
<br />
# start schema registry<br />
./bin/schema-registry-start ./etc/schema-registry/schema-registry.properties<br />
<br />
# start bottled water<br />
## I had to do this as user kafka_user<br />
kafka_user@raspberrypi:/opt/bottledwater-pg $ ./kafka/bottledwater --postgres=postgres://localhost<br />
-- or --<br />
runuser -l kafka_user -c"cd /opt/bottledwater-pg; ./kafka/bottledwater --postgres=postgres://localhost"<br />
<br />
#create table in postgresql (this creates a topic - which should be the same name as the table). Make sure it has a primary key.<br />
/usr/local/pgsql/bin/psql -Ukafka_user kafka_user -c "CREATE TABLE test_table_with_pk(id INT NOT NULL PRIMARY KEY, some_text VARCHAR, a_float_val FLOAT);"<br />
# list topics<br />
root@raspberrypi:/opt/confluent-3.0.1# bin/kafka-topics --list --zookeeper localhost:2181<br />
Java HotSpot(TM) Server VM warning: G1 GC is disabled in this release.<br />
_schemas<br />
test_table_with_pk<br />
<br />
#listen to topic<br />
root@raspb32:/opt/confluent-3.0.1# ./bin/kafka-avro-console-consumer
--topic table_with_pk --zookeeper localhost:2181 --property
print.key=true --from-beginning</blockquote>
<br />
Insert some data into your table, and you can see it in the consumer:<br />
<blockquote class="tr_bq">
terminal running postgres:<br />
<blockquote class="tr_bq">
kafka_user=# SELECT * FROM table_with_pk;<br />
id | test_text | a_float <br />
----+-------------+---------<br />
1 | some text 1 | 1.8<br />
2 | some text 2 | 2.8<br />
3 | some text 3 | 2.87609<br />
4 | test text 4 | 12<br />
(4 rows)<br />
<br />
kafka_user=# INSERT INTO table_with_pk VALUES(5,'test text5', 17.98);<br />
INSERT 0 1</blockquote>
<br />
terminal running a kafka consumer:<br />
<blockquote class="tr_bq">
root@raspb32:/opt/confluent-3.0.1# ./bin/kafka-avro-console-consumer --topic table_with_pk --zookeeper localhost:2181 --property print.key=true --from-beginning<br />
OpenJDK Zero VM warning: G1 GC is disabled in this release.<br />
...<br />
{"id":{"int":4}} {"id":{"int":4},"test_text":{"string":"test text 4"},"a_float":{"double":12.0}}<br />
{"id":{"int":5}} {"id":{"int":5},"test_text":{"string":"test text5"},"a_float":{"double":17.98}}</blockquote>
</blockquote>
Please let me know if you run into any problems - or especially if there are errors in this HOW-TO.<br />
<br />
Thanks.Mule's Earhttp://www.blogger.com/profile/01650109925841837119noreply@blogger.com0tag:blogger.com,1999:blog-8546198732026666862.post-78477675391232386302017-01-28T12:13:00.003-08:002017-06-20T10:41:05.639-07:00Installing the C AVRO library on Raspbian JessieReference: <a href="https://avro.apache.org/" target="_blank">Apache Avro</a><br />
<br />
git clone https://github.com/apache/avro.git<br />
<br />
<a name='more'></a><br /><br />
#c<br />
cd avro<br />
sudo apt-get install gcc cmake asciidoc source-highlight libjansson-dev<br />
cd lang/c<br />
mkdir build<br />
cd build/<br />
cmake .. -DCMAKE_INSTALL_PREFIX=$PREFIX -DCMAKE_BUILD_TYPE=RelWithDebInfo<br />
make<br />
make test<br />
sudo make install<br />
<br />
# c++<br />
<br />
sudo apt-get install libboost-filesystem-dev libboost-system-dev libboost-program-options-dev libboost-iostreams-dev g++ flex bison libboost-dev cmake <br />
cd lang/c++<br />
cmake -G "Unix Makefiles"<br />
make<br />
make installMule's Earhttp://www.blogger.com/profile/01650109925841837119noreply@blogger.com0tag:blogger.com,1999:blog-8546198732026666862.post-13349741086017214142017-01-28T11:48:00.000-08:002017-06-20T10:39:38.155-07:00Install postgresql from source an Raspberry Pi 3 Debian JessieReference: <a href="https://www.postgresql.org/docs/current/static/installation.html">https://www.postgresql.org/docs/current/static/installation.html</a><br />
<br />
Pretty sure I included all of the prerequisites:<br />
<br />
<a name='more'></a><br /><br />
sudo -i<br />
cd /usr/src/<br />
wget https://ftp.postgresql.org/pub/source/v9.6.1/postgresql-9.6.1.tar.bz2<br />
tar xf postgresql-9.6.1.tar.bz2<br />
cd postgresql-9.6.1<br />
apt-get install -y libreadline-dev libperl-dev libpython-dev libxml2-dev libssl-dev<br />
# become non-root user<br />
exit <br />
./configure --with-python --with-perl --with-openssl --with-libxml --enable-debug<br />
<br />
make<br />
su -i<br />
make install<br />
useradd -M postgres<br />
mkdir /usr/local/pgsql/data<br />
chown postgres /usr/local/pgsql/data<br />
su - postgres<br />
/usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data<br />
/usr/local/pgsql/bin/postgres -D /usr/local/pgsql/data > /tmp/postgres_logfile 2>&1 &<br />
/usr/local/pgsql/bin/createdbkafka_user<br />
/usr/local/pgsql/bin/psqlkafka_user<br />
<br />
Feel free to add that path (export PATH=$PATH:/usr/local/pgsql/bin/) to your .bashrc.Mule's Earhttp://www.blogger.com/profile/01650109925841837119noreply@blogger.com0tag:blogger.com,1999:blog-8546198732026666862.post-18295748011094130642017-01-28T11:31:00.000-08:002017-06-20T10:42:34.215-07:00Install zookeeper packageAs part of the Kafka streaming project, I installed zookeeper. Here's how I did that:<br />
<br />
Ref: <a href="https://zookeeper.apache.org/" target="_blank">Apache Zookeeper</a><br />
<br />
<a name='more'></a><br /><br />
wget http://apache.mirrors.pair.com/zookeeper/current/zookeeper-3.4.9.tar.gz<br />
tar xf zookeeper-3.4.9.tar.gz -C /opt/<br />
sudo cp /opt/zookeeper-3.4.9/conf/zoo_sample.cfg /opt/zookeeper-3.4.9/conf/zoo.cfg<br />
<br />
sudo /opt/zookeeper-3.4.9/bin/zkServer.sh start<br />
#ensure zk is running<br />
echo stat | nc 127.0.0.1 2181<br />
<br />
With nothing connecting to zookeeper, the output will look something like:<br />
<br />
Zookeeper version: 3.4.9-1757313, built on 08/23/2016 06:50 GMT<br />
Clients:<br />
/127.0.0.1:60706[0](queued=0,recved=1,sent=0)<br />
<br />
Latency min/avg/max: 0/0/0<br />
Received: 2<br />
Sent: 1<br />
Connections: 1<br />
Outstanding: 0<br />
Zxid: 0x0<br />
Mode: standalone<br />
Node count: 4Mule's Earhttp://www.blogger.com/profile/01650109925841837119noreply@blogger.com0tag:blogger.com,1999:blog-8546198732026666862.post-9418423780637357362015-06-26T18:10:00.000-07:002017-06-20T10:43:50.821-07:00Changing Out a Zookeeper Instance - Rolling restartChanging Out a Zookeeper Instance:<br />
<br />
We were recently tasked with changing out 3 (of 5) Zookeeper instances without disrupting the customer's environment. I set up a test bed using 5 independent zookeeper instances with a simulated game running on another.<br />
<br />
<a name='more'></a><br /><br />
After stopping the zookeeper instance (see Step #8) I changed the connection properties on my db servers and the game simulation. It didn't really help with the game simulation - that data was loaded at the start, but I had it in place for the next time the app was restarted.<br />
<br />
These are the steps that worked for me:<br />
<br />
<ol>
<li>Check status of all running zookeepers. Do the leader last! </li>
<li>Spin up a new zk instance.</li>
<li>Assign a higher myid to the new instance than the current cluster.</li>
<li>Copy the zoo.cfg from a running instance to the new one.</li>
<li>Add the new instance to the zoo.cfg.<br />e.g. <br />server.9999=10.0.0.100:2888:3888<br />...<br />server.9995=10.0.0.242:2888:3888<br />server.10000=10.0.0.154:2888:3888 #New zookeeper instance.</li>
<li>Start zookeeper on the new machine.</li>
<li>Ensure that it joins the quorum. "echo stat | nc 127.0.0.1 2181", or there are other ways to do that.</li>
<li>Stop the instance to be retired - or you can just stop zk on the instance.</li>
<li>On the new instance - comment out the retired instance and restart zookeeper.</li>
<li>Ensure that is accepting connections (see #6 above) on all of the running servers.</li>
<li>Remove the retired zookeeper and add the new zookeeper to the zoo.cfg of the next instance and restart.</li>
</ol>
Hope that works for you.<br />
<ol>
</ol>
Mule's Earhttp://www.blogger.com/profile/01650109925841837119noreply@blogger.com0tag:blogger.com,1999:blog-8546198732026666862.post-54606072577241658222013-12-03T11:35:00.000-08:002013-12-03T11:35:04.323-08:00Using my IPAD as an extended display using AirDisplay.<h2>
<b><span style="font-weight: normal;">Using my IPAD as an extended display using AirDisplay.</span></b></h2>
I couldn't use <a href="http://getairdisplay.com/" target="_blank">AirDisplay</a> at McDonald's or Starbuck's. The wi-fi blocked local traffic. So - this is my work-around. <br />
<br />
In a nutshell:<br />
<ul>
<li>Enabled Bluetooth on both and connected my IPAD to my Mac.</li>
<li>Started AirDisplay on both and connected.</li>
<li>When it started up, my Mac's display shrunk to the size of my IPAD. </li>
<ul>
<li>System preferences (on my Mac)</li>
<li>Displays</li>
<li>Arrangement - un-check 'Mirror Displays' and position the small window.</li>
</ul>
</ul>
Mule's Earhttp://www.blogger.com/profile/01650109925841837119noreply@blogger.com0tag:blogger.com,1999:blog-8546198732026666862.post-83404672713235849122013-02-10T12:42:00.000-08:002013-02-11T16:24:08.843-08:00Gentoo and CitrixThis is how I set-up icaclient to access <a href="http://www.ajubeo.com/" target="_blank">Ajubeo</a>, which uses a Citrix ZenApp server.<br />
<br />
First, of course, was to emerge net-misc/icaclient. The only applicable USE flag was nsplugin. I don't know if it is necessary or not, but I used it for gnash and Icedtea. So "echo "net-misc/icaclient nsplugin" >> /etc/portage/package.use/package.use<br />
<br />
Then, of course, I added the file to package.keyword with ~amd64. Emerge then tells you that you have to go the Citrix website. The message, from elogv: │Download the client RPM file x86? ( ICAClient-12.1.0-0.i386.rpm )<br />
│ amd64? ( ICAClient_12.1.0-0.x86_64.rpm ) from<br />
│&http://www.citrix.com/English/ss/download/details.asp?downloadId=2323812&productId=1689163 │<br />
and place it in /usr/portage/distfiles.<br />
<br />
Once you do that, you can emerge again, and it will extract the rpm, putting everything into /opt/Citrix. Unfortunately, this is not the end of my story.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj6JXRo2owufgLzN6A01Utp5X42Q4w1UAJ3DjzeVCfp7I3w2w2lbW4YR1OzQlNLiqa4O95j64N5MG6hw-PaIPrvDwQ75ZfT05VF6jD39vVPXL68OdpCOmw8Sp2_VaId7VNhA7szylXL22g/s1600/1.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="179" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj6JXRo2owufgLzN6A01Utp5X42Q4w1UAJ3DjzeVCfp7I3w2w2lbW4YR1OzQlNLiqa4O95j64N5MG6hw-PaIPrvDwQ75ZfT05VF6jD39vVPXL68OdpCOmw8Sp2_VaId7VNhA7szylXL22g/s320/1.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Choose <skip to logon></td><td class="tr-caption" style="text-align: center;"><br /></td></tr>
</tbody></table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjrdNJkKdeDJw4QqeEPnlcle_W_JLNtPxDDPUWPpnV4vkZppT52K6IxNLIeK_0RvRmTK8h3b3sGDYUeLHKAxnZBPvYSPinsW3mFsiHU2ercbI9iuu8cYDPn484OhI8MRdMZFsF2J0Ac-Hw/s1600/2.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="179" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjrdNJkKdeDJw4QqeEPnlcle_W_JLNtPxDDPUWPpnV4vkZppT52K6IxNLIeK_0RvRmTK8h3b3sGDYUeLHKAxnZBPvYSPinsW3mFsiHU2ercbI9iuu8cYDPn484OhI8MRdMZFsF2J0Ac-Hw/s320/2.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">I already had a username and password, so if you don't - you're on your own here.</td><td class="tr-caption" style="text-align: center;"><br /></td><td class="tr-caption" style="text-align: center;"><br /></td></tr>
<tr><td style="text-align: center;"></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><br /></td><td class="tr-caption" style="text-align: center;"><br /></td><td class="tr-caption" style="text-align: center;"><br /></td><td class="tr-caption" style="text-align: center;"><br /></td></tr>
</tbody></table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgKfMbkO0E5KLwguAjnqU5iUx36ri9CzTpDkzzdTa1uEf5o28ISHjNA2BkBl41Ypq-ob47a_Ydhl6Oi-HN7DtYlnrWkp2IAnbgJRRBbauR-RR8XA5PvqH1ozeV7b3fjQEYOuN-RR81Dcbw/s1600/3.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="179" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgKfMbkO0E5KLwguAjnqU5iUx36ri9CzTpDkzzdTa1uEf5o28ISHjNA2BkBl41Ypq-ob47a_Ydhl6Oi-HN7DtYlnrWkp2IAnbgJRRBbauR-RR8XA5PvqH1ozeV7b3fjQEYOuN-RR81Dcbw/s320/3.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Click on the link for the VCloud Portal - which is really a link for the ica client.</td><td class="tr-caption" style="text-align: center;"><br /></td><td class="tr-caption" style="text-align: center;"><br /></td></tr>
</tbody></table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEinqda3cPDNz-9zDEFG1jTJC_yWDXWwu3GLblottw4VKb_5I6v_MypHPVSvdnGG_zg0DanPjRHf_n1x0Os1U6HDaU5YBKAIjszjEARbOhQIQWBTBJ8TJSYNaKSwFSkOKKXHd8fo7NceHxk/s1600/4.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="179" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEinqda3cPDNz-9zDEFG1jTJC_yWDXWwu3GLblottw4VKb_5I6v_MypHPVSvdnGG_zg0DanPjRHf_n1x0Os1U6HDaU5YBKAIjszjEARbOhQIQWBTBJ8TJSYNaKSwFSkOKKXHd8fo7NceHxk/s320/4.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Firefox didn't know how to handle launch.ica. So, I chose open with, and selected "other"</td></tr>
</tbody></table>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgeWi5f-sn7fCrwiWCqdtgtR-vrzY5VNaL_dOVCcQjTDkMyJ9GZcoSg9uYrta-TY9_GHczbUJHFX2DmdMz2h9gkFBwgTLxXk3QtdReOlTrx0B7Tp4fkPF0QClVNxUYUM04LAcJy8RY6e6U/s1600/6.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="179" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgeWi5f-sn7fCrwiWCqdtgtR-vrzY5VNaL_dOVCcQjTDkMyJ9GZcoSg9uYrta-TY9_GHczbUJHFX2DmdMz2h9gkFBwgTLxXk3QtdReOlTrx0B7Tp4fkPF0QClVNxUYUM04LAcJy8RY6e6U/s320/6.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Then, I navigated to /opt/Citrix/ICAClient/</td></tr>
</tbody></table>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiHaE-C3Wx-Dok4_bDzKq-Y9H2128r-AOyNbIoSQsYmsXv91BylsLLvj-vFz78m_S4_liM-_tSA77dT2l1dBw0G_sUh0NM7Ie2otCBHGK0RlkH6MsolKMscnhGgoi7dQ_UWrZsrPCnFAdU/s1600/7.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="179" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiHaE-C3Wx-Dok4_bDzKq-Y9H2128r-AOyNbIoSQsYmsXv91BylsLLvj-vFz78m_S4_liM-_tSA77dT2l1dBw0G_sUh0NM7Ie2otCBHGK0RlkH6MsolKMscnhGgoi7dQ_UWrZsrPCnFAdU/s320/7.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">And selected 'wcfmgr.bin'. I also chose to "Do this automatically..."</td></tr>
</tbody></table>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgeveVpZVZAr4VvdseirilqIIDpM4e0Myy4cWx8EGaDbrOF_3VZzp96TQNsOQLtKp60KzRtlvpetzCkjFrVnR5-x4YN-jvOo9FSfDoee26sChPQx4hcQY04I8_36LUf9FcHfsgkQ6sxKTo/s1600/8.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="179" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgeveVpZVZAr4VvdseirilqIIDpM4e0Myy4cWx8EGaDbrOF_3VZzp96TQNsOQLtKp60KzRtlvpetzCkjFrVnR5-x4YN-jvOo9FSfDoee26sChPQx4hcQY04I8_36LUf9FcHfsgkQ6sxKTo/s320/8.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">It's still a little cranky. And when I chose to cancel the warning about being unable to connect..</td></tr>
</tbody></table>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjKIiBDRHrtt0Rau10HpcBJDdvs5iI4mJOK8tnVxKxadoIWIVV50mTEdkKo5nUcDWT4rLTRuTnD4S9SHPuDFknPiL9vDzF-WdoTtqK4raS2TjMAohSNSpGJPVkysQPOuBK7abl8g145RdI/s1600/8a.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="179" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjKIiBDRHrtt0Rau10HpcBJDdvs5iI4mJOK8tnVxKxadoIWIVV50mTEdkKo5nUcDWT4rLTRuTnD4S9SHPuDFknPiL9vDzF-WdoTtqK4raS2TjMAohSNSpGJPVkysQPOuBK7abl8g145RdI/s320/8a.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">And tried to connect anyway...</td></tr>
</tbody></table>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBTc1zDV00XMLAyPWDhvOIw4TKkEsdszKHsTGe5chFr6TbbfyFeKYKRj-HkSC2PeJLcZo0ejwKsJaL42ao0TJaH4tYEJakwKk3HAaY8tbzZrQAVuOuenVZW-KOUJ1l_LIXBaxAlxa3Uow/s1600/8b.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="179" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBTc1zDV00XMLAyPWDhvOIw4TKkEsdszKHsTGe5chFr6TbbfyFeKYKRj-HkSC2PeJLcZo0ejwKsJaL42ao0TJaH4tYEJakwKk3HAaY8tbzZrQAVuOuenVZW-KOUJ1l_LIXBaxAlxa3Uow/s320/8b.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">I was rewarded with an SSL error.</td></tr>
</tbody></table>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhR4IvUgudDifk_Nfb-yEPGyDdYfKYYVErlAvqeFlfGVDijDAG4YRp4qgUCUoqlR4JmFrq1JZjLtRCKjcgGUTpo-k4aDKB9qejYWGdO_CX-axckLbmRJOsrKDwu1sVHPn-kw4Vb0btvWdM/s1600/9.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="179" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhR4IvUgudDifk_Nfb-yEPGyDdYfKYYVErlAvqeFlfGVDijDAG4YRp4qgUCUoqlR4JmFrq1JZjLtRCKjcgGUTpo-k4aDKB9qejYWGdO_CX-axckLbmRJOsrKDwu1sVHPn-kw4Vb0btvWdM/s320/9.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">This time I tried clicked ok on the "Unable to connect.." warning and then clicked on the <Arrow> connect to selection button. Bingo</td></tr>
</tbody></table>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj2Pv8m3ZVDXJesXBXX3SWf1u0hXhbshJaFFXu_pu8Rm2-2BLNpkpzH6F7nwDkk8ASaJKr9HQoxYC9FFeQcMsMsW0bI5bSdPQFg95mo47faCkNoX5qE2DBrE-WDG1YO7cq-aYWXuzAFBJ4/s1600/10.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="179" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj2Pv8m3ZVDXJesXBXX3SWf1u0hXhbshJaFFXu_pu8Rm2-2BLNpkpzH6F7nwDkk8ASaJKr9HQoxYC9FFeQcMsMsW0bI5bSdPQFg95mo47faCkNoX5qE2DBrE-WDG1YO7cq-aYWXuzAFBJ4/s320/10.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Ajubeo's login page.</td><td class="tr-caption" style="text-align: center;"><br /></td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgeWi5f-sn7fCrwiWCqdtgtR-vrzY5VNaL_dOVCcQjTDkMyJ9GZcoSg9uYrta-TY9_GHczbUJHFX2DmdMz2h9gkFBwgTLxXk3QtdReOlTrx0B7Tp4fkPF0QClVNxUYUM04LAcJy8RY6e6U/s1600/6.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"></a></div>
<br />
<a href="http://www.ajubeo.com/" target="_blank">Ajubeo</a> has a great site, and I can't wait to dig into it. I just had to get past this first hurdle.Mule's Earhttp://www.blogger.com/profile/01650109925841837119noreply@blogger.com0tag:blogger.com,1999:blog-8546198732026666862.post-56849597140597043752012-12-01T13:20:00.000-08:002012-12-02T19:11:45.346-08:00Linux BasicsI thought I knew something about Linux. All confident in my knowledge. But I didn't know Jack - still don't. But as I was reading the LPIC (Linux Professional Institute Certification) Study Guide I learned a lot. A LOT! Holy moly that book kicks ass. Now I just want to share a little of what I learned.<br />
<a name='more'></a><br />
I'm still in the first chapter - where they're going over some of the command line tools. They go over cat, tac, less and more, expand and much more. Just the bash history tricks alone were worth the read.<br />
<br />
Some of the things I didn't know: Ctrl-R brings up a reverse search of bash history. While editing a line in bash history Ctrl-K deletes all text to the end of the line and Ctrl-X and Backspace deletes to the beginning of the line. Arguably the coolest command I came across was Ctrl-X Ctrl-E which edits the line in your editor (in my case it's vim). Cool, right!<br />
<br />
The book describes pipes, streams and redirection, then goes on with introductions to cat, tac, head, tail, join, paste, expand, unexpand, od, sort, split, tr, uniq, fmt, tr, pr, nl, tee, more, less, wc and cut. They are just getting ready to finish off the first chapter with regular expressions using grep and sed. What a great book. Yeah, man is good and all, but man pages seem to be written for people who write man pages, not regular people. This LPIC book is written for regular people - or people like me, at least.<br />
<br />
I think I'm going to read it again. So far, it's yielded this command: ps auxx | grep -i "$1"|grep -v 'grep\|find'| tr -s ' ' | cut -d ' ' -f 2 which I named findPID, made executable (chmod +x findPID) and stuck in my ~/bin directory (which btw is already in my $PATH). I wrote this because I was going to change my watch-emerge script to quit when emerge was done. If I know the PID of emerge - which is the end product of 'findPID emerge' - I can do: tail --pid=`ps auxx| grep -i emerge | grep -v 'grep\|watch' | tr -s ' ' | cut -d ' ' -f 2` -f /var/log/emerge.log. That's the best, most complicated command I've ever written.<br />
<br />
And one more command that I'll write here, just so it sticks in my head a little better. You can do $cat > file.txt to send standard input to file.txt. What used to be the DOS copy con command. I'm glad I found that book. Mule's Earhttp://www.blogger.com/profile/01650109925841837119noreply@blogger.com0tag:blogger.com,1999:blog-8546198732026666862.post-24992491061023380942012-10-26T11:10:00.000-07:002012-10-26T11:17:05.408-07:00Gentoo On My HP Part 3Things are going pretty well. Not well enough for me to change my GRUB2 default yet... Well, I guess I could.<br />
<br />
But I did run across some curious problems and their solutions. The first was starting X via xdm (instead of startx). SLiM would just get stuck - not starting anything but what looked like a twm session (less terminal window). When I made xdm be the default display manager by editing /etc/conf.d/xdm and changing DISPLAYMANAGER="slim" to "xdm", the result was a loop of xdm trying to start "LXDE" but just kicking it back to the login screen.<br />
<br />
Checking /var/log/xdm.log yields<br />
<blockquote class="tr_bq">
<b><code>xdm error (pid 2441): Dbus error: Unable to open session: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory<br />xdm error (pid 2441): console-kit-daemon not running?</code></b></blockquote>
The solution was to add dbus to the default runlevel and restart. <code>rc-update add dbus default</code> They left that out of the manuals, I guess because dbus got pulled in by something else - not sure what.<br />
<br />
As a bonus to adding dbus to the default runlevel, my MP3 player is now auto-mounted when I plug it in. (At least) Two birds with one stone!<br />
<br />
Also, I emerged the "murrine" packages - and that cleared up most of my icon problems. x11-themes/murrine-themes and x11-themes/gtk-engines-murrineMule's Earhttp://www.blogger.com/profile/01650109925841837119noreply@blogger.com0tag:blogger.com,1999:blog-8546198732026666862.post-42435120955485358562012-10-25T10:00:00.001-07:002012-11-14T12:48:47.984-08:00Gentoo on my HP - ContinuedI'm pretty sure that I got lucky when I configured the sound on this thing. It didn't work at first, but I'm sure that it's because I didn't have the right flags in make.conf. When I installed pysolfc, the sound worked and I'm pretty sure I did that before messing with the sound configuration in the kernel. And all I did there was to compile it as modules so I could figure out which ones I needed and which ones I didn't.<br />
<br />
I had written a bunch of simple scripts for my Dell that I sure miss now. Some of them I rewrote - the really easy ones like the one to watch the progress of the emerge or the progress of the fetch. "tail -f /var/log/emerge.log" - easy, like I said, but I always forget where they put the emerge.log - so it comes in handy. I had a really cool one that backed up imo important files like /usr/src/linux/.config and /etc/conf.d and stuff like that and tacked the date onto it. Now I'll have to figure out how to do that again.<br />
<br />
<a name='more'></a><br />
<br />
BTW - Here's my make.conf file:<br />
<blockquote class="tr_bq">
<code>
# These settings were set by the catalyst build script that automatically built this stage.<br /># Please consult /usr/share/portage/config/make.conf.example for a more detailed example.<br />## CHOST="x86_64-pc-linux-gnu"<br />## CFLAGS="-march=btver1 -mtune=btver1"<br />## CXXFLAGS="${CFLAGS}"<br /><br />CFLAGS="-O2 -pipe"<br />CXXFLAGS="${CFLAGS}"<br /><br /># WARNING: Changing your CHOST is not something that should be done lightly.<br /># Please consult http://www.gentoo.org/doc/en/change-chost.xml before changing.<br /><br />CHOST="x86_64-pc-linux-gnu"<br />MAKEOPTS="-j3"<br /><br />PORTAGE_ELOG_CLASSES="error warn log info"<br />PORT_LOGDIR="/var/log/portage"<br />PORTAGE_ELOG_SYSTEM="save"<br /><br />GENTOO_MIRRORS="http://mirror.usu.edu/mirrors/gentoo/ http://lug.mtu.edu/gentoo/"<br />SYNC="rsync://rsync.us.gentoo.org/gentoo-portage "<br /><br /># These are the USE flags that were used in addition to what is provided by the profile used for building.<br />#USE="mmx sse sse2"<br /><br />VIDEO_CARDS="radeon"<br />INPUT_DEVICES="evdev synaptics"<br /><br />ORIGINAL_USE="mmx sse sse2"<br />SYSTEM="-minimal bindist wifi vim-syntax offensive branding alsa libnotify startup-notification"<br />LANGUAGES="python tk gtk sqlite"<br />X="X udev -kde -gnome policykit consolekit dbus"<br />PICTURES="jpg jpeg png tiff apng"<br />VIDEO="xvid x264 mp4 mpeg theora matroska ffmpeg gstreamer"<br />AUDIO="aac flac mp3 ogg vorbis libass openal"MISC="pdf rar"<br /><br />USE="${ORIGINAL_USE} ${SYSTEM} ${LANGUAGES} ${X} ${PICTURES} ${VIDEO} ${AUDIO} ${MISC}"</code></blockquote>
I really like that way of doing my USE flags.<br />
<br />
I'm emerging libreoffice right now. They say it takes 8/(# of cores) hours to compile. It's going on 2 hours and the last I saw (maybe 1/2 hour ago) it was at like 63/104. It's in the middle of a big section right now, so I can't tell how far it is. I may have to go with the compiled version next time.<br />
<br />
There's still plenty to do. It's usable now, but not how I want it. But that's at least half the fun of <a href="http://www.gentoo.org/" target="_blank">Gentoo</a>. It's like a video game puzzle - only it's real life. When you're done - you have your computer <i>just</i> the way you like it.<br />
<br />
A quick list looks like this:<br />
<br />
Fix SLiM (doesn't work with xdm) I have to use startx.<br />
Fix icons - not all are showing up.<br />
Rewrite backup script.<br />
Install Java and add use flag and emerge -N world<br />
...<br />
<br />
Projects:<br />
Learn how to use python curses and write 2 apps- 1) to browse portage and extract descriptions from packages and 2) to clean old log files<br />
<br />
****************************************************************<br />
UPDATE: The libreoffice compile took 9 hours. Of course, I was using my computer the whole time (Firefox, Thunderbird, various others programs)- and it was noticeably slower - but assuredly usable.<br />
<br />
****************************************************************<br />
UPDATE2: The backup script is:<br />
#! /bin/bash<br /><br />date=$(date +"%Y%m%d")<br /><br />tar -cjf /home/all_users/backup$date.tbz2 /etc /usr/src/linux/.config /root/bin/ <br />
<br />
<br />Mule's Earhttp://www.blogger.com/profile/01650109925841837119noreply@blogger.com0tag:blogger.com,1999:blog-8546198732026666862.post-779750695170699482012-10-21T11:26:00.000-07:002012-10-22T20:55:17.171-07:00Gentoo on my HP2000-425NRWell, my Dell died :-( May it rest in peace. So I bought the cheapest good machine I could find, or maybe the best cheap machine - which turned out to be a HP-2000. Memory was on sale, so I got an extra 4G.<br />
<br />
Specs:<br />
AMD E300 1.3 GHz processor.<br />
6GB Ram<br />
320GB Hard Drive <br />
Ralink RT5390 Wireless<br />
ATI Technologies Inc AMD Radeon HD 6310 GraphicsATI<br />
ATI Technologies Inc SBx00 Azalia (Intel HDA) (rev 40)<br />
<br />
Installing Linux has proven problematic.<br />
<br />
<a name='more'></a><br />
<br />
Of course, I started with Gentoo. And, no doubt, I'll end with Gentoo. But in the middle... The Gentoo minimal CD does not come with support for my wireless card, and I'm not using the wired connection (because I'm using the free Internets at McDonald's) So, I decided that the easiest solution would be to go with a different distro.<br />
<br />
I tried Fedora Core (17) first, but it wouldn't re-partition my hard drive. It would not let me resize the Windows partition. So I used Ubuntu, which is as close to no-brain as one can get. It resized the Windows partition but <i>that </i>made Windows unusable. This is undoubtedly my fault, not sure what I did, but I suspect it had to do with the boot partition.<br />
<br />
So, I ended up installing FC17, redoing all the partitions, and re-installing Windows. The intermediate result was a working dual-boot computer.<br />
<br />
But FC is not for me. It's a solid distro - I don't want to say anything bad about it, but I miss Gentoo. Let's just say that FC17 installed and ran Zeitgeist without my knowledge or consent. Yeah, I should pay more attention, but when your downloading and installing hundred's of packages and their dependencies; How careful can you really be? And if you don't know why I'm bothered by zeitgeist, you may want to look into the whole Zeitgeist concept. <br />
<br />
I have to say this somewhere, so I'll say it here. FC17 often boots with a blank screen - like it didn't load the video module. I try switching to other tty's to no avail. I still haven't figured out why it does this or how to cure it. Sometimes a reboot works, sometimes not. But if I shut all the way down and restart (cold boot) it always works. I suspect something with grub - but no proof yet.<br />
<br />
**************************************************************************<br />
<br />
So, the question is how to get Gentoo on this machine.<br />
<br />
If there is an educational part to this post, I'd guess that this is it.<br />
<br />
I used gparted in FC17 to create a new partition for Gentoo and mounted it under a gentoo directory created in /mnt. Then, with FC running and connected to the internets I downloaded a stage3 tarball and portage snapshot and decompressed them onto that gentoo directory. I edited my config files in FC.<br />
<br />
I wrote a quick little script that I run as 'sudo' to get into the gentoo system from a terminal window in FC:<br />
<br />
<code>
#! /bin/bash <br />
mount /dev/sda9 /mnt/Gentoo </code><br />
<code>cp -L /etc/resolv.conf /mnt/Gentoo/etc/ </code><br />
<code>echo "--> Gentoo mounted"<br /><br />mount -t proc none /mnt/Gentoo/proc<br />echo "--> /proc mounted"<br /><br />mount --rbind /sys /mnt/Gentoo/sys<br />mount --rbind /dev /mnt/Gentoo/dev<br />echo "--> sys and dev mounted"</code><br />
<code><br />chroot /mnt/Gentoo /bin/bash</code><br />
<br />
Of course I chmod +x the file, and just 'su mountGentoo' when I'm ready to go. When I'm in the chrooted environment I do env-update && source /etc/profile and I'm good to go. I have the Gentoo environment and commands - I'm emerging left and right.<br />
<br />
More to come.<br />
<br />
************************************************************<br />
<br />
And, just to finish off this post - How I Solved Everything! I didn't solve it, the interwebs did. I just googled the right things.<br />
<br />
Apparently there is a file called linux-firmware that has the 'drivers' or firmware that allows the kernel to speak to the hardware. Unfortunately, I don't have the piece of paper upon which I wrote the error message which dmesg provided. But during one of my many sessions with the blank screen I logged-in (without visual feedback) and sent dmesg to a file. "Loading firmware file 'rt2860.bin'" and "Failed to request Firmware".<br />
<br />
So, once I emerged linux-firmware - the screen worked and my wireless worked! I was happy! Still am, come to think of it.<br />
<br />
Just one more quick thing. This morning, when I booted up, wlan0 did not function. The bootup error message was "Could not set interface wlan0 flags" and "Operation not possible due to RF-Kill". Turns out my wifi was turned off. I figured that one out on my own. <br />
<br />
When I installed linux-firmware there is an additional step that's needed before it can really be used. When you're configuring the kernel you actually have to enter text. I went <a href="http://forums.gentoo.org/viewtopic-t-880373-start-0.html" target="_blank">here</a> and towards the bottom, the guy spells it out for me.:<br />
<blockquote class="tr_bq">
Use no framebuffers with radeon/kms. quoting the guide: Quote:<br />Support for frame buffer devices ---> <br /> (Disable all drivers, including VGA, Intel, nVidia, and ATI) <br /><br /><br /> Sounds like the firmware blobs are not incorporated in the kernel config: Code:<br />emerge radeon-ucode<br /><br /><br />Code:<br />cd /usr/src/linux;make menuconfig<br /><br />Edit menuconfig <br /> Device Drivers ---> <br /> Generic Driver Options ---> Quote:<br />[*] Include in-kernel firmware blobs in kernel binary <br /> (radeon/PALM_me.bin radeon/PALM_pfp.bin radeon/SUMO_rlc.bin) External firmware blobs to build into the kernel binary <br /> (/lib/firmware) Firmware blobs root directory <br /><br /><br /><br /> Select "() External firmware blobs to build into the kernel binary" a dialog box opens. <br /> Copy and paste Quote:<br />radeon/PALM_me.bin radeon/PALM_pfp.bin radeon/SUMO_rlc.bin<br /><br /> onto the blank line in the dialog and enter. <br /> A new line will appear "(firmware) Firmware blobs root directory" Select it and another dialog appears. Backspace to clear it and type in or copy and paste Quote:<br />/lib/firmware</blockquote>
I was worried about having absolutely NO framebuffer support compiled into the kernel, or even as a module, but it sure worked!Mule's Earhttp://www.blogger.com/profile/01650109925841837119noreply@blogger.com0tag:blogger.com,1999:blog-8546198732026666862.post-59690065702859274842012-02-17T16:58:00.001-08:002012-02-26T08:57:05.452-08:00A cheesy application to check for new podcasts and transfer them to my Walkman#! /usr/bin/env python<br />
<br />
""" Version 0.02.02<br />
Just a simple file manipulation tool.<br />
Used in this instance as a Podcast updater.<br />
Still stuff to do - see TODO file"""<br />
<br />
import sys, os, time, stat, shutil<br />
import tkFileDialog, ttk, tkMessageBox<br />
from Tkinter import *<br />
<br />
<a name='more'></a><br />
<br />
class Application(Frame):<br />
<br />
import ConfigParser<br />
# Set these until settings are done - then set them with setup function<br />
sourceList = targetList = deviceList=[]<br />
targetList = []<br />
deviceList = []<br />
dirList = []<br />
workList = []<br />
<br />
ask = IntVar<br />
empty = IntVar<br />
statusVar = StringVar(Tk())<br />
sVar = StringVar()<br />
tVar = StringVar()<br />
dVar = StringVar()<br />
changeMade = FALSE<br />
now = time.time()<br />
then = 0<br />
home= os.environ['HOME']+'/'<br />
cfg = ConfigParser.RawConfigParser()<br />
<br />
def __init__(self,master=None):<br />
Frame.__init__(self, master)<br />
statusVar = StringVar()<br />
<br />
self.pack()<br />
statusVar = ""<br />
<br />
self.createWidgets()<br />
self.startup()<br />
self.refresh() <br />
<br />
<br />
def startup(self):<br />
pass<br />
#Check for config file<br />
self.cfg.read(self.home+"/.config/podcm.conf")<br />
<br />
tmpstring=(self.cfg.get('cbox','cboxSource'))<br />
self.sourceList=tmpstring.split(',')<br />
self.sourceCombobox ['values'] = self.sourceList<br />
<br />
tmpstring=(self.cfg.get('cbox','cboxTarget'))<br />
self.targetList=tmpstring.split(',')<br />
self.targetCombobox ['values'] = self.targetList<br />
<br />
tmpstring=(self.cfg.get('cbox','cboxDevice'))<br />
self.deviceList=tmpstring.split(',')<br />
self.deviceCombobox ['values'] = self.deviceList<br />
<br />
self.sourceCombobox.set(self.sourceList[0])<br />
self.targetCombobox.set(self.targetList[0])<br />
self.deviceCombobox.set(self.deviceList[0])<br />
<br />
tmp=(self.cfg.getboolean('chkbuttons','ask'))<br />
if tmp:<br />
self.askCheck.select()<br />
else:<br />
self.askCheck.deselect()<br />
<br />
tmp=(self.cfg.getboolean('chkbuttons','empty'))<br />
if tmp:<br />
self.emptyCheck.select()<br />
else:<br />
self.emptyCheck.deselect()<br />
<br />
self.then = (self.cfg.get('itime', 'last'))<br />
<br />
<br />
<br />
def refresh(self):<br />
pass<br />
<br />
self.sourceCombobox.set(self.sourceList[0])<br />
self.targetCombobox.set(self.targetList[0])<br />
self.deviceCombobox.set(self.deviceList[0])<br />
self.update()<br />
podDir = self.sourceCombobox.get()<br />
targetDir = self.targetCombobox.get()<br />
<br />
self.statusVar.set("Checking for new files...")<br />
<br />
for f in os.listdir(podDir):<br />
## check is DIR?<br />
g = os.path.join(podDir,f)<br />
mode = os.stat(g).st_mode<br />
t=0<br />
for h in os.stat(g):<br />
if (t == 9) and (self.now -h < 86400) and stat.S_ISDIR(mode) and (g != targetDir):<br />
##print (now - h,f)<br />
self.dirList.append(f)<br />
t=t+1<br />
self.txt.insert (END,"Folders altered in the last 24 hours in %s :\n" % podDir) <br />
<br />
for f in self.dirList:<br />
self.txt.insert(END,f+'\n')<br />
<br />
for f in self.dirList: ## for each directory in the dirList<br />
g = os.path.join(podDir,f) # append the name of the dir to the working pod dir<br />
##print ("try this:",os.listdir(g)[9]) -----> didn't work<br />
for h in os.listdir(g): # for each file in the directory<br />
j = os.path.join(g,h) # change g so it now is the path + file<br />
mode = os.stat(j).st_mode # get the type of file<br />
t = 0 # my counter because position 9 is itime<br />
## print("Try this #2 ",os.stat(j)[9]) ---> worked but...<br />
for i in os.stat(j):<br />
##print f,j,h,i<br />
if (t == 9) and (self.now - i < 86400) and (stat.S_ISREG(mode)):<br />
self.workList.append(j)<br />
t=t+1<br />
<br />
self.txt.insert(END,"\nNew Files:\n")<br />
<br />
for f in self.workList:<br />
self.txt.insert(END,f+'\n')<br />
<br />
self.statusVar.set("These are the files that have been changed in the last 24 hours.")<br />
<br />
<br />
def execute(self):<br />
pass<br />
podDir = self.sourceCombobox.get()<br />
targetDir = self.targetCombobox.get()<br />
<br />
<br />
if self.empty:<br />
pass<br />
self.statusVar.set("Clearing Target Directory.")<br />
for f in os.listdir(targetDir):<br />
g = os.path.join(targetDir,f)<br />
os.remove(g)<br />
<br />
self.update() <br />
<br />
self.statusVar.set("Copying Files Into Target Directory.")<br />
for f in self.workList:<br />
self.update()<br />
self.statusVar.set("Copying %s " %f)<br />
shutil.copy2(f,targetDir)<br />
<br />
<br />
self.statusVar.set("Copy Complete.")<br />
<br />
<br />
def save(self):<br />
pass<br />
print ("save function")<br />
##cfg.set(<br />
self.cfg.set('chkbuttons','ask',self.ask)<br />
self.cfg.set('chkbuttons','empty',self.empty)<br />
with open(self.home+"/.config/podcm.conf",'wb') as configfile:<br />
self.cfg.write(configfile)<br />
<br />
<br />
def end(self):<br />
pass<br />
if (self.changeMade):<br />
mb = tkMessageBox.askyesnocancel(<br />
message = "Do you wish to save your configuration before quitting?",<br />
icon = 'question',<br />
title = "Save current configuration?")<br />
<br />
if mb == None:<br />
return<br />
elif mb == True:<br />
self.save()<br />
elif mb == False:<br />
self.quit()<br />
<br />
else:<br />
self.destroy()<br />
self.quit()<br />
<br />
<br />
def copy2device(self):<br />
pass<br />
self.execute<br />
deviceDir = self.deviceCombobox.get()<br />
if (self.workList):<br />
for f in self.workList:<br />
self.update()<br />
self.statusVar.set("Copying %s " %f)<br />
shutil.copy2(f,deviceDir)<br />
self.statusVar.set("Copy Complete.")<br />
<br />
def sourceCboxChanged(self,t):<br />
## No idea why the second argument is required, but it will complain if it's not there<br />
self.changeMade = True<br />
<br />
def targetCboxChanged(self,t):<br />
self.changeMade = True<br />
<br />
def deviceCboxChanged(self,t):<br />
self.changeMade = True<br />
<br />
def askCheckChanged(self):<br />
self.changeMade = True<br />
self.ask=not(self.ask)<br />
<br />
<br />
def emptyCheckChanged(self):<br />
self.changeMade = True<br />
self.empty=not(self.empty)<br />
<br />
def source(self):<br />
pass<br />
## options defaultextension (string), filetypes (list), initialdir (string), initialfile (string)/<br />
## parent (widget), title(string)<br />
# adjust later so that list is only 5 deep<br />
b = tkFileDialog.askdirectory(initialdir=self.sourceCombobox.get())<br />
self.sourceList=[b]+self.sourceList<br />
self.sourceCombobox ['values'] = self.sourceList<br />
self.refresh()<br />
self.changeMade = TRUE<br />
<br />
def target(self):<br />
pass<br />
b = tkFileDialog.askdirectory(initialdir=self.targetCombobox.get())<br />
self.targetList=[b]+self.targetList<br />
self.targetCombobox ['values'] = self.targetList<br />
self.targetCombobox.current(0)<br />
self.changeMade = TRUE<br />
<br />
def device(self):<br />
pass<br />
b = tkFileDialog.askdirectory(initialdir=self.deviceCombobox.get())<br />
self.deviceList=[b]+self.deviceList<br />
self.deviceCombobox ['values'] = self.deviceList<br />
self.deviceCombobox.current(0)<br />
self.changeMade = TRUE<br />
<br />
def createWidgets(self):<br />
self.frame0 = Frame(self)<br />
self.frame0.pack()<br />
<br />
self.executeButton = Button(self.frame0, text ="Execute", command=self.execute)<br />
self.executeButton.grid(row=6, column=2,sticky=E)<br />
self.executeButton.focus_set()<br />
<br />
self.copy2deviceButton = Button(self.frame0, text="Copy to Device", command=self.copy2device)<br />
self.copy2deviceButton.grid(row=6, column=1)<br />
<br />
self.quitButton = Button(self.frame0, text ="Quit",command=self.end)<br />
self.quitButton.grid(row=6, column=3,sticky=E)<br />
<br />
self.saveButton = Button(self.frame0, text ="Save Preferences",command=self.save)<br />
self.saveButton.grid(row=6, column=0)<br />
<br />
self.sourceButton = Button(self.frame0, text = "Change Source", command=self.source)<br />
self.sourceButton.grid(row=0, column=3)<br />
<br />
self.targetButton = Button(self.frame0, text ="Change Target", command=self.target)<br />
self.targetButton.grid(row=1, column=3)<br />
<br />
self.deviceButton = Button(self.frame0, text ="Change Device", command=self.device)<br />
self.deviceButton.grid(row=2, column=3)<br />
<br />
self.sourceCombobox = ttk.Combobox(self.frame0,width=50, textvar=self.sVar, values=self.sourceList)<br />
self.sourceCombobox.grid(row=0,column=0,columnspan=3,sticky=W)<br />
self.sourceCombobox.bind('<<ComboboxSelected>>', self.sourceCboxChanged)<br />
<br />
self.targetCombobox = ttk.Combobox(self.frame0,width=50, textvar=self.tVar, values=self.targetList)<br />
self.targetCombobox.grid(row=1,column=0,columnspan=3,sticky=W)<br />
self.targetCombobox.bind('<<ComboboxSelected>>', self.targetCboxChanged)<br />
<br />
self.deviceCombobox = ttk.Combobox(self.frame0,width=50, textvar=self.dVar, values=self.deviceList)<br />
self.deviceCombobox.grid(row=2,column=0,columnspan=3,sticky=W)<br />
self.deviceCombobox.bind('<<ComboboxSelected>>', self.deviceCboxChanged)<br />
<br />
self.askCheck = Checkbutton(self.frame0, text="Don't ask next time",<br />
command = self.askCheckChanged)<br />
self.askCheck.grid(row=3, column=0, columnspan=2,sticky=W)<br />
<br />
self.emptyCheck = Checkbutton(self.frame0, text="Empty target directory first?", variable=self.empty,<br />
command = self.emptyCheckChanged)<br />
self.emptyCheck.grid(row=4, column=0, columnspan=2,sticky=W)<br />
<br />
self.txt = Text(self.frame0)<br />
self.txt.grid(row=5,columnspan=4)<br />
<br />
self.status=Label(self.frame0,textvariable=self.statusVar,height=2,wraplength=560,justify=LEFT)<br />
self.status.grid(row=7,columnspan=4)<br />
<br />
## Start here<br />
app=Application()<br />
app.master.title("Podcast Mover")<br />
app.master.geometry("600x580+20+20")<br />
<br />
app.mainloop()<br />
<br />
****************************************************<br />
This was my first attempt at using settings (ConfigParser) so a config file is required<br />
for the application to function. If there is no config file, the values should have defaults like ~.<br />
<br />
And the ~/.config/podcm.conf file<br />
<br />
[cbox]<br />
<br />
cboxSource : /home/panchod/Audio/<br />
<br />
cboxTarget : /home/panchod/Audio/0Today<br />
<br />
cboxDevice : /media/WALKMAN/MUSIC/<br />
<br />
<br />
[chkbuttons]<br />
<br />
ask = TRUE<br />
empty = FALSE<br />
<br />
[itime]<br />
<br />
last : "00000000000"Mule's Earhttp://www.blogger.com/profile/01650109925841837119noreply@blogger.com0tag:blogger.com,1999:blog-8546198732026666862.post-90460677620768172502012-02-13T12:11:00.000-08:002012-02-13T15:10:52.435-08:00Python Tkinter and Feedparser.py Cleaned Up a Little.So, after some massaging of the available information and examples, I now have a list (not necessarily a python list) of items in the feed. Instead of printing them, I'm going to insert them into the form.<br />
<a name='more'></a><br />
<br />
At first, I was going to use a text box. While that is functional, I see the need, in the future, to be able to select a feed item for download. So, for now I'm going to use a listbox. I'll add this to the GUI file and reference it as listbox1 in the program.<br />
<br />
Some of the big changes: The button no longer has a function associated with it. I moved the she-bang line (#! /usr/bin/env python) from the GUI file to the program as it should have been from the start.<br />
<br />
listing of generic3.py:<br />
<br />
import sys<br />
from Tkinter import *<br />
<br />
class Application(Frame):<br />
<br />
def __init__(self,master=None):<br />
Frame.__init__(self, master)<br />
self.pack()<br />
self.createWidgets()<br />
<br />
def createWidgets(self):<br />
<br />
self.frame0 = Frame(self)<br />
self.frame0.pack()<br />
<br />
self.label1 = Label(self.frame0, text="This is a text label.")<br />
self.label1.grid( row=0, column=0)<br />
<br />
self.listbox1 = Listbox(self.frame0,width=60)<br />
self.listbox1.grid (row=2,column =0, columnspan =3)<br />
<br />
self.button1 = Button(self.frame0, text = "Push Button")<br />
self.button1.grid(row=1,column=0)<br />
<br />
self.button2 = Button(self.frame0, text = "Quit",command=self.quit)<br />
self.button2.grid(row=1,column=1)<br />
<br />
<br />
And the listing for l-gen3.py<br />
<br />
#! /usr/bin/env python<br />
<br />
from generic3 import *<br />
import urllib2, sys, feedparser<br />
<br />
app=Application()<br />
app.master.title("Hello There")<br />
app.master.geometry("600x400+20+20")<br />
<br />
#GLOBALS<br />
url1="http://feeds.feedburner.com/antiwar-radio"<br />
<br />
def feedinfo(url, output=sys.stdout):<br />
<br />
feed_data = feedparser.parse(url)<br />
channel, items = feed_data.feed, feed_data.entries<br />
<br />
<br />
print("Feed Title: ",channel['title'])<br />
<br />
for item in items:<br />
print( "Title: ",item['title'])<br />
app.listbox1.insert(END, item['title'])<br />
<br />
return<br />
<br />
def process(self):<br />
pass<br />
print ("process activated")<br />
print (url1)<br />
feedinfo(url1)<br />
print (data)<br />
<br />
feedinfo(url1)<br />
#app.button1.bind("<Button-1>", process)<br />
<br />
app.mainloop()<br />
<br />
Next step, I guess, is to select one of the feeds in the list and download it.Mule's Earhttp://www.blogger.com/profile/01650109925841837119noreply@blogger.com0tag:blogger.com,1999:blog-8546198732026666862.post-75385914775322269452012-02-12T10:14:00.000-08:002012-02-13T08:30:29.376-08:00Using FeedParser.py to Get Feed Info From a Website.This is what I'm talking about with the documentation. I wanted to open the url and get the information from it. But how do you do that? I tried <tt class="descclassname">urllib2.</tt><tt class="descname">urlopen</tt><big>(</big><i>url[, data][, timeout]</i><big>), but that didn't work. </big>FeedParser.py comes with a nice example program - and that is what I shamelessly copied.<br />
<a name='more'></a><br />
<br />
Now, when you press the button, this program reads the feed information and prints it out.<br />
<br />
Next step will be to populate a text box or combo box with the data from the feed.<br />
<br />
# -*- coding: utf-8 -*-<br />
from generic2 import *<br />
import urllib2, sys, feedparser<br />
<br />
#List of uples (label, property-tag, truncation)<br />
COMMON_CHANNEL_PROPERTIES = [<br />
('Channel title:', 'title', None),<br />
('Channel description:', 'description', 100),<br />
('Channel URL:', 'link', None),<br />
]<br />
<br />
COMMON_ITEM_PROPERTIES = [<br />
('Item title:', 'title', None),<br />
('Item description:', 'description', 100),<br />
('Item URL:', 'link', None),<br />
]<br />
<br />
INDENT = u' '*4<br />
<br />
<br />
app=Application()<br />
app.master.title("Hello There")<br />
app.master.geometry("600x400+20+20")<br />
<br />
#GLOBALS<br />
url1="http://feeds.feedburner.com/antiwar-radio"<br />
data=""<br />
<br />
def feedinfo(url, output=sys.stdout):<br />
"""<br />
Read an RSS or Atom feed from the given URL and output a feed<br />
report with all the key data<br />
"""<br />
feed_data = feedparser.parse(url)<br />
channel, items = feed_data.feed, feed_data.entries<br />
#Display core feed data<br />
for label, prop, trunc in COMMON_CHANNEL_PROPERTIES:<br />
value = channel[prop]<br />
if trunc:<br />
value = value[:trunc] + u'...'<br />
print >> output, label, value<br />
print >> output<br />
print >> output, "Feed items:"<br />
for item in items:<br />
for label, prop, trunc in COMMON_ITEM_PROPERTIES:<br />
value = item[prop]<br />
if trunc:<br />
value = value[:trunc] + u'...'<br />
print >> output, INDENT, label, value<br />
print >> output, INDENT, u'---'<br />
return<br />
<br />
def process(self):<br />
pass<br />
print ("process activated")<br />
print (url1)<br />
feedinfo(url1)<br />
print (data)<br />
<br />
<br />
app.button1.bind("<Button-1>", process)<br />
<br />
app.update()<br />
<br />
app.mainloop()<br />
<br />
<u>UPDATE: </u><br />
This is how it works for me. The guys who wrote feedparser.py and the example did a great job. And once I realized that feedparser results in a dictionary type, it was fairly easy to extract the data I wanted.<br />
<br />
Here's the revised program:<br />
<br />
<br />
from generic2 import *<br />
import urllib2, sys, feedparser<br />
<br />
app=Application()<br />
app.master.title("Hello There")<br />
app.master.geometry("600x400+20+20")<br />
<br />
#GLOBALS<br />
url1="http://feeds.feedburner.com/antiwar-radio"<br />
data=""<br />
<br />
def feedinfo(url, output=sys.stdout):<br />
<br />
feed_data = feedparser.parse(url)<br />
channel, items = feed_data.feed, feed_data.entries<br />
<br />
<br />
print("Feed Title: ",channel['title'])<br />
<br />
for item in items:<br />
print( "Title: ",item['title'])<br />
<br />
return<br />
<br />
def process(self):<br />
<br />
pass<br />
print ("process activated")<br />
print (url1)<br />
feedinfo(url1)<br />
print (data)<br />
<br />
app.button1.bind("<Button-1>", process)<br />
<br />
app.update()<br />
<br />
app.mainloop()Mule's Earhttp://www.blogger.com/profile/01650109925841837119noreply@blogger.com0tag:blogger.com,1999:blog-8546198732026666862.post-49381614396895001782012-02-10T20:45:00.001-08:002012-02-11T17:28:00.652-08:00A Way to Use Tkinter and PythonThere's probably as many ways to learn as there are people.<br />
<br />
I'm learning Python, and OOP, and Java and a ton of other things. There seems to be no end to the number of things of which I am ignorant. But I like Python - it fits my procedural style while giving me a transition into OOP.<br />
<br />
There are a ton of great sites out there. <a href="http://effbot.org/tkinterbook/" target="_blank">Effbot</a> has a kick-ass site. Python's <a href="http://docs.python.org/index.html" target="_blank">documentation</a> is very good. But my questions are often more practical than the intelligent stuff these guys talk about.<br />
<br />
For example, this my first post, is all about writing the code for a Tkinter application. But I wanted to put the code for the GUI in one file and all the meat in another. Seems simple enough, and for most of the programmers out there it undoubtedly is. It's probably so easy, they don't feel the need to write about it. I, on the other hand, am baffled by easy stuff like this.<br />
<br />
And so, I wrote a really simple application to do - well not much of anything. But it does contain a file which defines the application - two buttons and a text label. The other file contains an instance of the class and a function which reacts to a button press. I hope this helps someone.<br />
<br />
file listing for generic.py - the gui portion of the project. So the idea is to write the GUI part - and then to more-or-less forget about it.<br />
<br />
<u>#! /usr/bin/env python</u><br />
<br />
"""generic.py was written as an example of how to split a project<br />
into multiple files"""<br />
<br />
import sys<br />
from Tkinter import *<br />
<br />
class Application(Frame):<br />
<br />
def __init__(self,master=None):<br />
Frame.__init__(self, master)<br />
self.pack()<br />
self.createWidgets()<br />
<br />
def createWidgets(self):<br />
<br />
self.frame0 = Frame(self)<br />
self.frame0.pack()<br />
<br />
self.label1 = Label(self.frame0, text="This is a text label.")<br />
self.label1.grid( row=0, column=0)<br />
<br />
self.button1 = Button(self.frame0, text = "Push Button")<br />
self.button1.grid(row=1,column=0)<br />
<br />
self.button2 = Button(self.frame0, text = "Quit" , command = self.quit)<br />
self.button2.grid(row=1,column=1)<br />
<br />
<u>And, its companion l-gen.py</u><br />
<br />
<br />
#! /usr/bin/env python<br />
from generic import *<br />
<br />
app=Application()<br />
app.master.title("Hello There")<br />
app.master.geometry("600x400+20+20")<br />
<br />
def process(self):<br />
pass<br />
print ("process activated")<br />
<br />
app.button1.bind("<Button-1>", process)<br />
<br />
app.mainloop()<br />
<br />
Next, I want to make the button go to a website. Where I'm going is to get a podcast feed list, and eventually i want to download them and play them. For now, I'll use Tkinter, but I'd prefer to use PyQt. I can't wait to figure out how to do that.Mule's Earhttp://www.blogger.com/profile/01650109925841837119noreply@blogger.com0