Home Automation - Raspberry Pi controlled Power Strip

Watch my previous post on a prototype project using Raspberry Pi. Now you would know what was that prototype for.

Yes, I now have built a power strip that is completely controllable by a Raspberry Pi over network. 

The idea of controlling a power strip over network is nothing new, but this is certainly a unique one and I needed this badly.  I have built a power strip with more features than the off-the-shelf ones and they were at least twice as expensive as what it took me to build this one. I can fix any problem in this device from software to a relay to a resistor -- in case any of these is blown. Oh yeah, the happiness of building something yourself is invaluable :)

To me, one of the driving factors for such a device is: We have Tata Sky+ HD at home and we record TV programs left and right. It has almost reached a point that we don't watch live content except news (for the time comfort and sake of skipping ads). Many interesting series do come in the night, for which, we leave the set-top-box (STB) on -- naturally for prolonged time (till morning) even if the recording is just for an hour or so in the midnight. Tata Sky+ HD STB consumes 20+W of power even on standby. I wanted to avoid this waste of power, by having the ability to turn on/off the power sockets using a rather low power device.

Raspberry Pi consumes just around 3.5W and has complete networking support. With abundant GPIO pins, I can use it to control at least 8 different sockets. This is one of the main reasons for me to choose a Raspberry Pi. While I had worked on AVR based micro controllers, but getting a AVR board on the network would have been much more difficult than this. Also the power of Linux, provides numerous software that were of great use to me while building this (python, apache2, flask etc.,)

Some of the key features:

• Power Sockets are controllable via a web interface, so available from any device on the network.
• Web service supports REST based interaction for easy integration with any app that I might write in the future (at least for the android phones).
• Core web and daemon logic on python that avoids cross-compilation work for Rs Pi architecture. Rewrite code easily and just drop it for deployment. This was awesome! God bless interpretors!
• Ability to control sockets from the Internet once I VPN into my home network. (this should let us do the recording even when we are on vacation; how cool is it that I can turn a socket in Bangalore on or off, from anywhere in the world)
• Timer support, so I can schedule a On or Off event after a while. Say, I can turn on a socket at midnight 12am, run it for an hour and turn it off! There is a dedicated power daemon that runs in the background to take care of scheduling these requests when its time.
• The power sockets are still controllable using their dedicated switches, so you don't have to hunt for a browser to turn on the sockets.

Here is the power strip in action:

I have also now installed the power strip for the real purpose I wanted for. See the demo where I turn OFF my Tata Sky+ HD Set Top Box using the mobile.

I have taken photos all the way while I built this power strip. Stay tuned for lot more technical details (electronic circuits, hardware, software) with photos, on what went behind this power strip.

Sneak peek:

 

MyApp: Calendar Widget for Android 2.2 (Froyo)

I had been looking for a decent home screen widget for the calendar(s) on my Samsung Galaxy S (currently on Froyo). Could not find one that is free, so wrote one.

Features:
* Includes events from all your calendars on the phone (personal, corporate etc., etc).
* Support for recurring events.
* Separate section for today's events and later events.
* Auto refresh in every 2 minutes (No, it doesn't wake up the phone, if sleeping -- so don't worry about battery).
* Occupies 3 (rows) x4 (columns) cells in home screen.
* Shows up to 6 events at a time.

Note:
* Apparently Google has deprecated the Calendar ContentProvider post Froyo. So this widget may not work post Froyo (including 2.2.1, I suppose). I only have a 2.2 phone and the emulators don't support Calendar (deprecated) -- so no idea.
* Tested only on Samsung Galaxy S on Froyo.

When I get a firmware update to Gingerbread to my SGS next month (hopefully!), I shall try to port this to Gingerbread as well, if possible.

Download the .apk installer here or point your phone to this QR code to download directly



Developed and built using Eclipse Indigo on Ubuntu.

MyApp: Spb Wallet to KeePass convertor

Spb Wallet is one of the best wallet applications that I've used across various mobile platforms. However, it isn't available on all mobile platforms and it isn't free either. Once I moved on to Android, I hit a road-block because Spb Wallet isn't available for Android (yet). I'm sure you would understand how painful it is to not have a wallet app, once you are used to.

KeePass is the alternate. KeePass is an open-source software for password management. To be fair, KeePass isn't as great as Spb Wallet, but does its job. Being open-source, it is available on almost all platforms including desktops.

The pain here is the conversion. I have tonnes of data on Spb Wallet that manually entering them on KeePass is a no-go. Unfortunately, and mostly intentionally, Spb Wallet doesn't export to any well known format for import into KeePass. There weren't any handy tools to convert either. Thankfully Spb Wallet had a text export and KeePass had many import options. But it isn't directly compatible, because Spb Wallet doesn't have any proper structure to the exported TXT file and the grammar is quite ambiguous. KeePass has a well defined XML structure for import (found it by doing a sample XML export from KeePass). I wrote this python script to convert the Spb Wallet TXT export file into the XML format that KeePass can understand. In reality, Spb Wallet has more "specific" fields than KeePass, so there isn't always a direct mapping. Any non-mappable field (Account Number for example) will be appended in the notes section of KeePass so no information is lost.

This script is a simple parser that understands and converts the Spb Wallet TXT export file. It maintains the internal state of parsing and learns dynamically about what the current token is -- some times even looking ahead into the file to resolve ambiguities.

This script is probably not so robust on errors. But this did the job for my Wallet export which is reasonably big.

If you find any bug on this script that you want me to fix, report here. I *MAY* fix it for you.

Here is the source:

#
# Code to Convert Spb Wallet TXT export file to KeePass XML import file.
#
# Original Author: Gerald Naveen A (http://geraldnaveen.blogspot.com)
#
# License:
# You are free to modify/distribute/use for commercial/non-commercial/
# personal applications. Any modification to this code needn't be published. 
# However, any publication of this code or the derivative of this code, should 
# include the original author and this license text.
#
import sys

def mywrite(f, str):
f.write("{0}\n".format(str));
def main():
print "\nSpb Wallet to KeePass Convertor v 1.0 by Gerald Naveen\n";
print "Report bugs at http://geraldnaveen.blogspot.com/2010/11/myapp-spb-wallet-to-keepass-convertor.html\n";
if len(sys.argv) < 3:
 print "Usage: spb_wallet_to_keepass.py <spb_txt_export_file> <keepass_xml_import_file>";
 print "\nWhere,\nspb_txt_export_file: path to the TXT file exported from Spb Wallet.";
 print "keepass_txt_import_file: path to the output XML file, that shall be imported into KeePass.";
 return;
try:
 ifile = open (sys.argv[1], 'r');
except:
 print "Could not open input file", sys.argv[1];
 return;

try:
 ofile = open (sys.argv[2], 'w');
except:
 print "Could not open output file", sys.argv[2];
 return;

FOLDER_NAME_TOKEN=1;
ENTRY_NAME_TOKEN=FOLDER_NAME_TOKEN+1;
BEFORE_NOTES_TOKEN=ENTRY_NAME_TOKEN+1;
NOTES_TOKEN=BEFORE_NOTES_TOKEN+1;
INVALID_VALUE='invalid';

next_token=ENTRY_NAME_TOKEN;
folder_name = INVALID_VALUE;
entry_name = INVALID_VALUE;
user_name = INVALID_VALUE;
password = INVALID_VALUE;
url = INVALID_VALUE;
notes = INVALID_VALUE;
valid_entry = False;

mywrite(ofile, '<?xml version="1.0" encoding="utf-8" standalone="yes"?>');
mywrite(ofile, '<pwlist>');
try:
 for line in ifile:
  line = line.strip('\r\n');
  if len(line) == 0:
   # empty line
   if valid_entry == False:
    # entry name found after folder name
    folder_name = entry_name;
    entry_name = INVALID_VALUE;
   else:
    # found the last line of the entry..dump
    mywrite(ofile, '<pwentry>');
    if folder_name != INVALID_VALUE:
     mywrite(ofile, '<group>{0}</group>'.format(folder_name));
    mywrite(ofile, '<title>{0}</title>'.format(entry_name));    
    if user_name != INVALID_VALUE:
     mywrite(ofile, '<username>{0}</username>'.format(user_name));
    if password != INVALID_VALUE:
     mywrite(ofile, '<password>{0}</password>'.format(password));
    if url != INVALID_VALUE:
     mywrite(ofile, '<url>{0}</url>'.format(url));
    if notes != INVALID_VALUE:
     notes=notes.replace('\n', '&#xD;&#xA;');
     mywrite(ofile, '<notes>{0}</notes>'.format(notes));
    mywrite(ofile, '</pwentry>');
    user_name = INVALID_VALUE;
    password = INVALID_VALUE;
    url = INVALID_VALUE;
    notes = INVALID_VALUE;
   valid_entry = False;
   next_token=ENTRY_NAME_TOKEN;
  else:
   if next_token == ENTRY_NAME_TOKEN:
    entry_name = line;
    next_token = BEFORE_NOTES_TOKEN;
   else:
    valid_entry = True;
    if next_token == BEFORE_NOTES_TOKEN:
     if line.startswith('User Name:'):
      user_name = line[len('User Name:'):].strip(' ');
     elif line.startswith('Password:'):
      password = line[len('Password:'):].strip(' ');
     elif line.startswith('Web Site:'):
      url = line[len('Web Site:'):].strip(' ');
     elif line.startswith('Notes:'):
      if notes == INVALID_VALUE:
       notes = line[len('Notes:'):].strip(' ');
      else:
       notes += '\n' + line[len('Notes:'):].strip(' ');
      next_token = NOTES_TOKEN;
     else:
      # any unknown params should go as notes.
      if notes == INVALID_VALUE:
       notes = line;
      else:
       notes += '\n' + line;
    elif next_token == NOTES_TOKEN:
     # any thing from the notes section.
     notes += '\n' + line;
except:
  print "Unknown error occured while processing the input file.";
mywrite(ofile, '</pwlist>');    
ifile.close();
ofile.close();
print "Success. Now import {0} in KeePass as KeePass XML".format(sys.argv[2]);
if __name__ == "__main__":
   main()

Download spb_wallet_to_keepass.py

Update [26-Nov-2010]:

If the script ran successfully, but the output XML file didn't work on import, it could most likely be a CRLF issue. Try this in such cases:

Lets assume your file is test.txt.

1. Open test.txt in Notepad
2. "Save as" another file, say test2.txt (Note: select Utf8 as Encoding).
3. Use test2.txt as input

Caller Location Info v 0.3 for WinMo

Licensed under

Here is the next version of my Caller Location Info app for Windows Mobile (for India).

Release-notes:
1. Includes a bunch of new additions to the mobile numbering. At least 250-300 new numbers added.
2. Includes 2 new service providers - Tata Docomo and Loop Mobile.
3. No changes to the STD list.
4. No bug fixes (no known bugs actually :D)

The installation instructions and other properties remain the same. See the earlier post for that.

Download the CAB installer.

Enjoy!

TVProgramGuide -- developer's view - #2

This is a follow up of my previous post on developer's view on my TVProgramGuide application.

There were 2 APIs identified : InitIR and GetIRCode

The next step is to identify the return types, calling convention, parameter list and types of these two. Let me explain how I discovered them for one of those APIs (GetIRcode - the difficult one) by reverse-engineering their disassembly.

Disassembly of ThatDll!GetIRCode

10001190 83ec0c          sub     esp,0Ch
10001193 8d442400        lea     eax,[esp]
10001197 53              push    ebx
10001198 8d4c2408        lea     ecx,[esp+8]
1000119c 33db            xor     ebx,ebx
1000119e 50              push    eax
1000119f 51              push    ecx
100011a0 c6442410c0      mov     byte ptr [esp+10h],0C0h
100011a5 c644241102      mov     byte ptr [esp+11h],2
100011aa 885c2412        mov     byte ptr [esp+12h],bl
100011ae 885c2413        mov     byte ptr [esp+13h],bl
100011b2 885c2414        mov     byte ptr [esp+14h],bl
100011b6 885c2415        mov     byte ptr [esp+15h],bl
100011ba c644241601      mov     byte ptr [esp+16h],1
100011bf 885c2417        mov     byte ptr [esp+17h],bl
100011c3 c744240c08000000 mov     dword ptr [esp+0Ch],8
100011cb e870ffffff      call    ThatDll!SendVendorCmd (10001140)
100011d0 3bc3            cmp     eax,ebx
100011d2 5b              pop     ebx
100011d3 8b542410        mov     edx,dword ptr [esp+10h]
100011d7 750c            jne     ThatDll!GetIRCode+0x55 (100011e5)
100011d9 8a4c2404        mov     cl,byte ptr [esp+4]
100011dd 880a            mov     byte ptr [edx],cl
100011df 83c40c          add     esp,0Ch
100011e2 c20400          ret     4
100011e5 c602ff          mov     byte ptr [edx],0FFh
100011e8 83c40c          add     esp,0Ch
100011eb c20400          ret     4

Calling convention:
An easy technique to identify the calling convention is to look for the 'ret' statement @25,28 (I would also advise to double check with the caller's next instruction disassembly to make sure he doesn't play with the stack pointer). In stdcall calling convention the callee is supposed to free the stack space for arguments. I think here we are debating only over stdcall and cdecl calling conventions. So, if the 'ret' statement has any value given as operand (no. of bytes to free up on stack), then the calling convention should be 'stdcall'. In most cases, DLLs are stdcalls -- and this observation ensures that for this dll.

Return type (and out params):
In this case, we had already discovered that the technical return type is an int (and was returning 0x0 on keypress and 0xff on no keypress). However, we are still lacking the keycode when a key was pressed. If you look at the epilogue of the function, there are two branchings (clearly two rets @25,28). Please note the "mov byte ptr [edx],0FFh" @26. This looks like an error case, when SendVendorCmd failed (@21). A close look at the diassembly (@18,21) reveals that this code flow occurs when the return value of SendVendorCmd is non-zero (!=ebx); it should also be noted that the return value of GetIRCode is the same as the return value of SendVendorCmd (note that there is no change in eax after the call to SendVendorCmd). If you look at the success path(@22,23), an out parameter of SendVendorCmd (@[esp+4]) is copied on to the address in edx (note the byte ptr mov -- the out param value is so an unsigned char).

Argument list and types:
We are almost done. The only missing piece is to figure out what is edx pointing to. This is a crucial and challenging part. Please bear with me. The statement 'mov edx, dword ptr[esp+10h]'@20, means that the address of the parameter is on the stack. The return statement denotes a 4 byte cleanup on the stack; so it is likely that the fucntion takes only one parameter and that is a pointer to a byte (unsigned char*). However, it is not clear if the [esp+10h] belongs to the local stack variable in this function or is really a argument pushed by the caller -- use of ebp might have been much clearer, but we don't have a choice here. Looking at the disassembly of SendVendorCmd (ret 8) tells me that it uses 8 bytes on stack for arguments. So after the call to SendVendorCmd, the esp will be less by 8 bytes. Now if you carefully account for all the push and pop instructions in this function before [esp+10h]@20, you would find out that the [esp+10h] is indeed pointing to [esp_0+4h] if esp_0 is the esp at the time of entry of the function. [esp+4h] at the entry point clearly skips the return address and lands on the first argument to the function.

And hence the function is 'int (__stdcall *ThatDll!GetIRCode)(unsigned char*)'

I believe I don't have to mention about the InitIR API. But that was pretty simple; the prototype turned out to be 'void (__stdcall *InitIR)(void)' :D.

Now, how to I use ths info to dynamically hook onto the existing TV tuner application is the only critical missing part. Stay tuned!

TVProgramGuide -- developer's view

=== this post is for educational purposes only. please do not apply these concepts to hack into or do illegal stuff ===

As promised earlier, here is my post on what's behind my TVProgramGuide application.

For the ones who do not have the background on the topic and for the ones who did not read my post on my application -- I had a TV tuner hardware and an application that works with it. I could use my TV tuner remote to control the TV. Now I need to find a way to hook into this design and capture the TV remote key presses, so I can use it in my own applications (important: without affecting the TV app's functionality). I'm keen in mentioning only the critical and difficult portions of this app.

The whole issue can be split into multiple major issues:

1. Finding out the DLL and the APIs that the current app uses to read the remote key presses.
2. Reverse-engineer the APIs and find out their calling convention, return types and parameter lists (and types) -- you would definitely need if you are going to hook into the APIs.
3. Find a means to hook/patch the functionality to allow both the TV app and my app to capture the strokes -- multiple options available. Read on.
4. Decipher the codes to match the real keys on remote -- if 1-3 are complete, this shouldn't be difficult.

Let me talk about each one of them in detail.

Finding out the DLL and APIs:

The TV application and the tuner hardware are from different vendors. This have me the hope that there might mostly like be a dll which provides the set of APIs to talk between the two. Using dependency-walker I found the list of modules that the TV tuner application was depending on. I filtered a set of non-system DLLs that got installed along with the TV tuner application. Then listed down the "exports" table from each of those DLLs and looked for any reasonably named API that relates to this. In one of those DLLs (I'm not disclosing the name of the DLL to keep this hack anonymous), there was an API named 'GetIRCode' -- having known that remotes work on Infra-Red (IR), this was suspicious. There were other APIs named 'InitIR', 'GetOneButtonStatus' etc., which seemed more and more closer towards the functionality I was looking at. I was almost sure.

Here's the export table of that identified DLL :



To make sure if these are indeed the APIs that I was looking for: I attached the debugger (windbg) to the TV tuner application and added breakpoints to all APIs in that DLL. As the application starts, I got a breakpoint hit in Dll!InitIR. Makes sense. Then, I could see breakpoints continuously being hit on Dll!GetIRCode (yes, continuously). I just realized that there was no callback mechanism, and the application continuously polls for keypresses by calling GetIRCode (ahem!, waste of CPU). But is it really what I think? Just to make sure that this API was doing something useful on a key press, I looked at the return code of GetIRCode after each invocation. It returned 0xff (likely a -1 in signed byte) most times. I set a conditional breakpoint on the return statement of this function to break if the return value is != 0xff (ie., break if register eax != 0xff). I realized from my testing that, whenever I pressed a key on my TV remote, this break point was hit and the return value was 0 -- hmmm, almost there, but where is the key code??? hack isn't that easy :). A good news was that, during the runtime (when I tested with remote key presses), the TV tuner application did not call any other API on this DLL.

So, at the end of this step, I have discovered the DLL and two APIs that I might need to hook into. And also that GetIRCode returns 0 once after a key was pressed (note: I still do not know how to identify the key, just hoping that this API would help) -- no idea about the calling convention, return types, the parameters I need to pass in to these APIs and their types. Way to go!!

Step 2 for reverse-engineering those APIs for calling convention, parameter list/types is a long topic, stay tuned!

TVProgramGuide - my application for tv tuners

This is about an application that I've recently developed, that can show real-time TV programme guide lines on the fly over a TV tuner video.

It is really interesting to see how subtle things can make a big difference in the way we carry out every day life. This idea struck my mind few weeks back, when I landed at entertainment.oneindia.in accidentally while trying to find out what movies are played for the day. The idea was to integrate this info about TV programmes into the existing TV tuner application, so I can fetch them whenever I need them. When I was first thinking about this idea, I didn't really expect it to be soo useful -- today I just can't watch TV without the aid of this app.

The idea sounds interesting but it is as vague as a patent. I spent the first week thinking about the feasibilty of this application, and about how to integrate this app into the TV system. I was pretty clear that this app is going to be of no-use if I cannot provide a means to use this application wirelessly (yes, using the TV remote). If I have to come to the comp, and use the mouse to find out 'what's coming up next?', I would rather visit the website in my browser and know about it. It is as simple as having a bookmark in my browser.

But I wasn't sure if I can hook into the TV remote, and get its signals. Even if I can, I should also ensure that my hooking does not affect the normal functioning of the TV tuner application, that is already using the TV remote and reacting to the signals. As specified in my earlier post on my TV tuner, the tuner's hardware (Trident) and the TV tuner application (Honestech) seem to be from different vendors -- this gave me the hope that there should be an interface available somewhere (although unpublished) using which the existing TV tuner app is receiving the TV remote key presses. I will definitely be writing a separate post on the technical details of how and what disassembling I had to do; but for now, it is that I've managed to discover the undocumented APIs that are used internally, and the appropriate DLLs and managed to hook into them seamlessly so the TV tuner application has no idea that I'm hooked in.

This hooking was done in C/C++. The remaining task was to download the TV programmes from their website (note: oneindia does not have the list for all channels. So I had to do a generic design to support any website; with abstract classes and interfaces, this isn't a problem anyway). For any non-system programming I prefer python (if not UI related) and C# if it has an associated UI. I admire the power of the modules/class libraries that these platforms provide; awesome! .Net comes handy with Http classes (System.Net.HttpWebRequest) to handle the HTTP requests/responses for downloading the programmes. And I used RegEx (System.Text.RegularExpressions.RegEx) to parse the HTML output and extract the program schedule. With a number of choices for sharing info (remote key press) between the C/C++ hook and the C# interface, I chose the simplest one: The windows registry. I intend to post a developer post on this later, but that's the overall technology behind this app.

Important features:

1. Automatically shows the 'now playing' programme on every channel change. The 'now playing' item is picked based on the current channel no. and the current time. This window shows up for 10 seconds and auto-hides. Very useful when we glance through the channels.
2. Channel change is detected by monitoring the remote key presses including numbers, up, down, recall etc., Interesting part was that channels can be changed by a sequence of key presses eg., key 1 followed by key 2 followed by key 3 in a short interval, is not 3 different channel changes, but a single change to channel no. 123. It had to be handled differently.
3. A special mode can be entered by a special key combination (that does nothing to the tuner app) in the TV remote. In this mode, the app overlays the 'coming up next 5' programmes list over the video. This info does not auto-hide. It can be closed by the Mute key on TV remote.
4. Seamless integration, so TV tuner works just well as before.

Here is the video showing my application in action as I watch TV on my comp:



Overall, I'm very happy with this app; one of my applications that I use the most. I'm hopeful to publish this app soon, after I make some generalizations (currently channel associations are hard-coded and not all channels are supported).

Caller Location Info v 0.2 for WinMo

Licensed under

[Update: 29-Aug-2009] A recent version of this app is available. See here.

Here is the next version of my Caller Location Info app for Windows Mobile (for India).

After I had release v 0.1, there was too many requests for making this app work for countries outside India and for landlines. Unfortunately, it is not feasible to implement a "similar" approach for most countries outside India. Please note that I do not claim that it is not possible at all. So when I got time, I extended the app for showing the city names for landlines -- based on the STD codes in India.

The installation instructions and other properties remain the same. See the earlier post for that.

Snapshots: (v 0.2)







Download the CAB installer.

Enjoy!

Caller Location Info v 0.1 for WinMo

== I had developed this application in Sep 2008; posting it here to give a context to my other applications and future versions of this app ==

Licensed under

This application for Windows Mobile 5+ displays the service provider and the location to which the caller (on incoming call) or receiver (on outgoing calls) belongs. The application hooks into the Windows Mobile state notification infrastructure and the caller info is automatically shown whenever there is an incoming call or outgoing call.

This application only works for mobile numbers in India -- sorry I don't have information for other countries; rather it does not seem feasible in many countries (including the US). In fact many countries now allow the users to carry their existing number to other service providers. I heard such a scheme is on its way in India too.

This app should work on any phone running WinMo 5 or above. I've tested it only on my ASUS p320.

How To:

After installation just click on the shortcut from Programs. That's it. I've purposely not installed the shortcut in Startup folder to give full-control to the user to choose whether or not to run the app.

Sadly, there is no easy way to exit the app other than rebooting your phone - if you have applications like process explorer, you can terminate the process at your will.

Enjoy the app!

Snapshots:





Download the CAB installer.

Note:

• It is possible that few recently allocated mobile numbers are not known to this app and will be reported as Unknown.
• Also this version only supports location information for mobile phones (not for landline std codes).
• The information might be wrong if the caller is roaming.