Home Automation: Arduino controlled Geyser

A long-pending post. I had built this project way back in May 2014, during the summer (in India). In one aspect, it seems better to have delayed this post, as my project has gone through a few bug fixes and enhancements, both in hardware & software -- so I can talk about the most recent version.

I had done a few other home automation stuff earlier like this one : DIY: Raspberry Pi controlled Power Strip - Part 1. I was/am constantly on the look out for any opportunity to automate things at home. We have been wasting lots of energy, as our geyser almost runs till cut-off most of the times and we won't need so much hot water during summer. This was simply because we won't remember to turn it off on time. I wanted to fix this by building a timer controlled switch for the geyser using Arduino.

I had earlier used Raspberry Pi for home automation. This time, I wanted to use Arduino for a few reasons. Unlike the power strip, which is completely network-controlled, this is a low power project and doesn't necessarily be network-controlled (I could actually snap in a Ethernet Shield for Arduino and make this network-controllable -- remember, I have Ethernet running over my power lines anyway, so easy to get a network anywhere). I also wanted to explore Arduino as part of this, as Arduino is relatively low cost and very low power (< 1W). I buy Arduino/Raspberry Pi from Ebay India directly (though they could be cheaper by other channels).

Here is the circuit:

This circuit was drawn using some online tool (that isn't perfect). The tool had bugs, so some of it didn't come out the way I want. Still conveys the idea.

The project is primarily, a relay (Relay2) driving the high power geyser (ours is around 2500W). The original plan was to drive this relay by a signal from Arduino. The high power relay that I procured, required a signal voltage of around 9V without which it couldn't really turn on the load. Arduino GPIO pins operate only at 5V. So I had to introduce another relay (Relay1) to supply the required 9V (external source), but on signal from Arduino. This also ensures, not much current is drawn directly from Arduino. Arduino Uno has a built-in voltage regulator (safe up to 12V), but I decided to be safe and use a custom voltage regulator using LM7805 (I wouldn't want to heat up Arduino or burn it -- lot cheaper to build an external voltage regulator - around Rs.70). I also have a 16x2 LCD to display the status/timer (see photos). The LCD is driven using the standard Arduino LCD library.

Here is the voltage regulator (built separately and tested out):

It doesn't turn on the geyser instantly. It runs a 10 second timer before it turns it on. This is to ensure that any electricity interruptions don't turn on/off the geyser too quickly. Better for the geyser.

 

Showing the countdown to shutdown:

 

 

The project in action (for the last 6 months). The USB cable connects to the data port of Arduino via a hole in the case -- this is used for software upgrades in-place; just connect my Mac and click on a button to flash it instantly with new code. eg., Once the winter started, I had to patch it to increase the timer a bit to get it to the right temperature.

This board is in series with the geyser, so turning on the physical switch doesn't turn on the geyser (expected).

Fixing a bricked TP Link Ethernet-over-Power Adapter

Though I have Wifi coverage throughout my home, for better bandwidth and lower latency/packet loss, I also run a Ethernet over the electric line (230VAC) using the Ethernet-over-Power (EoP) adapters. I have been using this for few years for high-speed connectivity across different rooms without having to lay new Ethernet cables. (eg., my home theatre system connects to my NAS via EoP to play HD videos without jitter over network).

This is the exact model I use:

http://www.tp-link.com.au/products/details/?model=TL-PA211KIT

One of them went dead last year, so I had to buy 2 more to provide enough coverage. They aren't that cheap and aren't available in India directly (though I could import via ebay or amazon). Few months back, another one went dead. When I say dead, it would not power on, when you connect to the power line; no LEDs will glow and will be functionally dead as well. 

This wasn't scaling (I can't keep buying new ones) and I didn't know what was wrong so I can prevent this. As it was anyways dead I decided to break it open and figure out what had happened (maybe just a fuse blown?). That's where it all started.

Unfortunately I don't seem to have photos during the disassembly (not sure why I didn't shoot). This was one of the hardest disassembly ever, for me. It is meant not to be opened. There is one screw at the back (hidden under a sticker). Unscrewing that doesn't do much, although required. The packaging is very rigid, you can't even break it easily. I drilled a small hole on this, using a Bosch drilling machine to peek in a bit :D yes, it was a risky thing to do. The white cover is locked on to the black case with notches in the sides -- I had to peek into the the heat vents to figure this out.  Using a thin screw driver as wedge, I could open the white cover revealing the mother board inside.

It runs of a proper ARM-based Atheros chip, along with a (expected) RealTek chip for Ethernet support. I could only see the top of the board, and most of the board's soldering was not accessible at this point, so I couldn't test any of the circuitry for faulty parts. I had to take the board out. Be careful if you are doing this -- as I figured out later that the board had been glued to the black case below; so it wouldn't come off the case easily. You need to apply force along the sides and take it off. There is no other screw, I can tell you now (this was my biggest scare, that if I miss a screw, the force might break the board).

With quite some struggle and care, I took the board off the case. This is how the back of the board looks:

Arrows in yellow, show the gum that was holding the board to the case. At first, I even thought if this was some sort of leakage from the underlying components.

This is how the top side of the board looks (yellow wire was soldered by me to test the board outside):

The arrow on the left points to the fuse. I tested for continuity and it looked ok. Then started testing  each capacitor. The 3 capacitors at the bottom right were seeming to be faulty (short on DC). I was a bit surprised to see all 3 capacitors being blown -- on further investigation, they were in parallel in the circuit and even one faulty capacitor could project all 3 to be faulty. When I looked at the back side of the board (red-rectangle as in the backside-view picture), I could see some leakage on one of the capacitors. The capacitor also had a slight bulge at its top (no photo). I was more hopeful then. On soldering out the capacitor, off the board, the other two capacitors tested normal -- Good!

This was the faulty capacitor:

Now I need to get a spare cap with the same spec. Ebay India came to rescue. Ordered 10 capacitors 1500uF/10V and were delivered in 3 days.

The above top-side picture actually shows the board with the new capacitor replaced. Packed it in and did one final round of testing before I packed it into its box.

Test success. And here it is the final working version back in action:

And that's how I fixed a TP link EoP adapter for Rs.10 :)

If you have one such dead one, give it a try. It is likely this cap issue.

Fixing Raspberry Pi filesystem corruption

While Raspberry Pi is a computer in its core, it is treated like an embedded device in most cases -- we can't expect a proper shutdown of Pi in most cases. This results in corruption of the / (root) file system (ext4) due to unclean unmount. 

In case you have been reinstalling Pi on to the sdcard every time this happens, you don't really have to reinstall. In most cases, mount the card via a card reader and run 

sudo fsck.ext4 /dev/xxxx  [xxxx the device file of the root partition]

(I assume linux by default). This should fix the / file system and the card will let your Pi boot now. If this doesn't help, try fixing the /boot partition:

sudo fsck.vfat  /dev/xxxx  [xxxx - the device file of the boot partition]

That said, you don't really have to do this. The clean and guaranteed protection against this problem could be provided by mounting the filesystem as read-only. Thus the filesystem is completely intact and will have no chance of corruption. In most cases, this should be fine (unless you have a reason to persist something on to the root filesystem). If you still need to write something to disk, write it to a different partition so it doesn't prevent Pi from booting if that goes corrupted.

How to mark the / and /boot partitions read-only:

You could do this from Pi itself, or on a different machine. Just edit /etc/fstab and add 'ro' in the flags as shown below:

/dev/mmcblk0p1  /boot       vfat    defaults,ro                0       2
/dev/mmcblk0p2  /           ext4    defaults,ro,noatime        0       1

Now reboot the Pi and it should boot with both these mount points in read-only mode. You could do a mount to verify, it should look something similar to this:

/dev/root on / type ext4 (ro,noatime,data=ordered)
devtmpfs on /dev type devtmpfs (rw,relatime,size=216108k,nr_inodes=54027,mode=755)
tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=44876k,mode=755)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /run/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,size=89740k)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620)
/dev/mmcblk0p1 on /boot type vfat (ro,relatime,fmask=0022, dmask=0022, codepage=cp437, iocharset=ascii,shortname=mixed, errors=remount-ro)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,relatime,size=89740k)

Note: Some services (like apache2) might fail to start, because they can't write to disk. You might have to check individual services to figure out what they are trying to write and resolve it appropriately. For eg., apache2 writes log files /var/log/apache2. It is best to disable logging. If there is not too much logging, you could map the log folder to /tmp (tmpfs -- as shown in mount).

as root:

mkdir /tmp/apache2 
chmod a+wx /tmp/apache2 
cd /var/log

rm -rf apache2

ln -s /tmp/apache2 apache2

There is no worry of file system corruption any more. You could switch off Pi any time. I have been using this setup (in my PowerStrip Project) for many months now, without issues.

SSH to Raspberry Pi first time without keyboard/monitor

Applies to "wheezy" version, not sure about other versions.

 

Raspberry Pi is mostly used as a remote machine and usually not tied up with any keyboard or a monitor, as in my case. Whenever you reimage a version of Raspberry Pi, connecting through ssh doesn't work.

Raspberry Pi's latest images come with "sshd" enabled by default. However, during first boot, "raspi-config" script (from /etc/profile.d/raspi-config.sh) kicks in and throws up a UI on the screen for the user to respond. This doesn't timeout and doesn't let the boot sequence complete; you would notice that Rsbpi has joined the network, but one cannot ssh to it (as the startup is stuck at this script and sshd is not started yet).

Workaround:

Once you reimage the SD card, mount the ext4 partition just created on the card. This will give access to the root (/) filesystem of the "wheezy" linux. Comment out the following line from /etc/inittab on that / filesystem:

1:2345:respawn:/bin/login -f root tty1

/dev/tty1 2>&1 # RPICFG_TO_DISABLE

This disables auto-login of root and raspi-config doesn't run until an interactive login happens as root. Now you can boot the SD card on Rsbpi and you should be able to ssh to it. 

Worked for me!

DIY: Raspberry Pi controlled Power Strip - Part 1

This post will cover some behind-the-scene details on what went behind making this power strip. Had been quite busy for a while, so couldn't make it earlier.

There were few questions that I had to answer before I was sure I could build this:

1. How do I control 220V AC from an electronic circuit, safely? Can I do it right the first time so I don't blow up the raspberry Pi? Being completely a software guy, this was a challenge. This was the first part of the problem I solved as shown in my earlier post on Controlling 220v Bulb using Raspberry Pi.
2. Can I fit this whole solution seamlessly into an existing power strip so it has a clean form-factor and intuitive to use? Finding one such power strip which had enough space to hold additional circuits was a challenge.
3. Building software so I can remotely control the power strip over any device. I decided to go the web interface route, so I can do it easily from any device. The software I eventually implemented is based on RESTful APIs, so it would also be easier for me to write an Android app or any other app over this interface.

Controlling 220V AC from Raspberry Pi

 

One of the foremost requirements for choosing Raspberry Pi was that it has programmable GPIO (General Purpose Input / Output) headers, without which any electronic interfacing would be pretty difficult and inefficient. Most micro-controllers (including modern microprocessors do provide GPIO pins) for interfacing with other low-level hardware peripherals. The voltage on a GPIO PIN can be controlled by instructing the micro-controller. Usually a TTL low/high is used to signal 0 or 1. But it is totally up to the interface to decide how to interpret it. Raspberry Pi uses 3V3 TTL, which means a 3.3V for high and 0V for low. Specially when you are using the GPIO pins for input, make sure the voltage doesn't exceed 3.3V. For the power strip, it is only in output mode.

Relay is another important component. It is an electro-magnetic switch that is used to turn on or off the high voltage. Relays require slightly higher voltages (>=12V) to work. When there is enough current flow, an electro-magnetic coil gets magnetized and pulls off a lever to turn on the switch. When the power is switched off, there is no magnetic field, and a mechanical spring pulls it back to its original position. Relay is a mechanical device and might suffer some latency and noise during its operation. It is not meant for high-precision control, but in my case this is good enough.

Since relays work on higher voltages and that it requires substantial current, a relay cannot be driven directly from a Raspberry Pi. It is very common to use a transistor as a switching device to turn on slightly higher voltages and/or when you need more current. This allows us to withdraw very less current (only the base current) from the controlling source (in our case Raspberry Pi) and to use a completely different power source (>=12V) for the controlled device (in our case, a relay). I have plugged in a 12V power adaptor which provides sufficient current to turn on/off the relay.

This is the circuit that drives a single relay: (The core idea of this circuit is a very common circuit that is used to control a relay via a transistor. I have customized it to appeal for the given use case.)

Remember that Raspberry Pi runs off USB power. Model B has a cap of 700mA in the inlet and this current is used for the complete functioning of Raspberry Pi. Any current that we draw out of this, is expensive and we need to be cautious. Specially if you are planning to drive a load (say a LED) directly, without using a different power source. 

WARNING: As I learned, there is no fuse of any sort behind the GPIO headers -- so any incorrect use might blow the micro-controller, thereby making a Raspberry Pi brick. Take extreme precautions and ensure that you are fine with the current you draw/sink from/into the GPIO.

The GPIO18 is just one of the many GPIO pins that could drive this transistor. To turn it on, my code will set a HIGH on that pin, thereby raising the voltage on that pin to 3.3V. With the required V-BE (the base-emitter voltage) at 0.7V to turn on, for the 4.7KOhm base resistor, the transistor will just draw around 0.5mA current from the raspberry Pi. Even when multiple relays are ON, there is no risk of over-drawing current from Raspberry Pi. The Diode D1 provides protection against any reverse current that could occur at the moment the relay is switched off.

Raspberry Pi has a GPIO python library that we can use to control the GPIO pins with ease. By being able to control from python, I didn't have to go through the pain of cross-compiling every time I modified any code.

Snip of code:

#
#initialization
#
import RPi.GPIO as GPIO
GPIO.setmode(GPIO.BCM) # use BCM pin numbering
GPIO.setup(gpio_pin_number, GPIO.OUT) # mark for output

#end of initialization; following could be called multiple times.

GPIO.output(gpio_pin_number, GPIO.HIGH) # output high / 3.3v
GPIO.output(gpio_pin_number, GPIO.LOW) # output low / 0v
#

As simple as that. Based on the command issued remotely, my power strip daemon would choose the right gpio_pin_number and set it to HIGH or LOW appropriately.

This post is getting long :). So, more about the packaging of the relay board, associated software design, in the subsequent posts.

Home Automation - Raspberry Pi controlled Power Strip

Watch my previous post on a prototype project using Raspberry Pi. Now you would know what was that prototype for.

Yes, I now have built a power strip that is completely controllable by a Raspberry Pi over network. 

The idea of controlling a power strip over network is nothing new, but this is certainly a unique one and I needed this badly.  I have built a power strip with more features than the off-the-shelf ones and they were at least twice as expensive as what it took me to build this one. I can fix any problem in this device from software to a relay to a resistor -- in case any of these is blown. Oh yeah, the happiness of building something yourself is invaluable :)

To me, one of the driving factors for such a device is: We have Tata Sky+ HD at home and we record TV programs left and right. It has almost reached a point that we don't watch live content except news (for the time comfort and sake of skipping ads). Many interesting series do come in the night, for which, we leave the set-top-box (STB) on -- naturally for prolonged time (till morning) even if the recording is just for an hour or so in the midnight. Tata Sky+ HD STB consumes 20+W of power even on standby. I wanted to avoid this waste of power, by having the ability to turn on/off the power sockets using a rather low power device.

Raspberry Pi consumes just around 3.5W and has complete networking support. With abundant GPIO pins, I can use it to control at least 8 different sockets. This is one of the main reasons for me to choose a Raspberry Pi. While I had worked on AVR based micro controllers, but getting a AVR board on the network would have been much more difficult than this. Also the power of Linux, provides numerous software that were of great use to me while building this (python, apache2, flask etc.,)

Some of the key features:

• Power Sockets are controllable via a web interface, so available from any device on the network.
• Web service supports REST based interaction for easy integration with any app that I might write in the future (at least for the android phones).
• Core web and daemon logic on python that avoids cross-compilation work for Rs Pi architecture. Rewrite code easily and just drop it for deployment. This was awesome! God bless interpretors!
• Ability to control sockets from the Internet once I VPN into my home network. (this should let us do the recording even when we are on vacation; how cool is it that I can turn a socket in Bangalore on or off, from anywhere in the world)
• Timer support, so I can schedule a On or Off event after a while. Say, I can turn on a socket at midnight 12am, run it for an hour and turn it off! There is a dedicated power daemon that runs in the background to take care of scheduling these requests when its time.
• The power sockets are still controllable using their dedicated switches, so you don't have to hunt for a browser to turn on the sockets.

Here is the power strip in action:

I have also now installed the power strip for the real purpose I wanted for. See the demo where I turn OFF my Tata Sky+ HD Set Top Box using the mobile.

I have taken photos all the way while I built this power strip. Stay tuned for lot more technical details (electronic circuits, hardware, software) with photos, on what went behind this power strip.

Sneak peek:

 

Controlling 220v Bulb using Raspberry Pi

I have always had this excitement to control high-voltage devices from electronics. I could never do one because of lack of hardware and/or the scare for dealing with high voltage and getting it right the first time. Finally, I now have a proof of concept project that controls a 220v bulb from a Raspberry Pi. More details, maybe in a later post. Right now, I need to build a better actual project with this :) 

Raspberry Pi runs on Raspbian Wheezy linux and the controlling code is written in Python.

WDTV Live -- Firmware Hacking Series -- Part 3

Patching custom rootfs onto existing firmware

Once you have the serial cable built as mentioned in my previous post, it is possible to patch the current firmware installation with custom rootfs, without having to upgrade/reinstall the whole firmware with custom one.

These are the high-level steps that we need to do:

1. Discover the partition table and identify where the rootfs is stored in the flash storage.
2. Read the rootfs of the current firmware and push it to a different machine.
3. Modify the rootfs to your needs, on the other machine.
4. Upload it back to WDTV and write the flash partition appropriately.

Discovering the rootfs partition

~ # mount
rootfs on / type rootfs (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
/dev/sigmblockh on / type cramfs (ro)
none on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
usb on /proc/bus/usb type usbfs (rw)
mdev on /dev type tmpfs (rw)
devpts on /dev/pts type devpts (rw)
none on /tmp type tmpfs (rw)
/dev/loop0 on /tmp/static_config type minix (rw)

Note the highlighted line. /dev/sigmblockh device is the one mounted as / partition. To confirm print the first 100 bytes on the device and you could notice the CRAMFS header.

~ # hexdump -c -n 100 /dev/sigmblockh

0000000   E   = 315   (  \0   @ 021 003 003  \0  \0  \0  \0  \0  \0  \0
0000010   C   o   m   p   r   e   s   s   e   d       R   O   M   F   S
0000020 234 333 016 324  \0  \0  \0  \0   k   &  \0  \0 313 016  \0  \0
0000030   C   o   m   p   r   e   s   s   e   d  \0  \0  \0  \0  \0  \0
0000040 355   A 354 003   D 001  \0   d 300 004  \0  \0 355   A 354 003
0000050 354  \v  \0   d 001 031  \0  \0   b   i   n  \0 377 241 354 003
0000060  \t  \0  \0   d                                                
0000064
~ # 

We still need to know the exact flash disk and the partition to which this device points to.

~ # ls /dev/sigm* -l
brw-rw----    1 root     root     254,   0 Jan  1  2000 /dev/sigmblocka
brw-rw----    1 root     root     254,   1 Jan  1  2000 /dev/sigmblockb
brw-rw----    1 root     root     254,   2 Jan  1  2000 /dev/sigmblockc
brw-rw----    1 root     root     254,   3 Jan  1  2000 /dev/sigmblockd
brw-rw----    1 root     root     254,   4 Jan  1  2000 /dev/sigmblocke
brw-rw----    1 root     root     254,   5 Jan  1  2000 /dev/sigmblockf
brw-rw----    1 root     root     254,   6 Jan  1  2000 /dev/sigmblockg
brw-rw----    1 root     root     254,   7 Jan  1  2000 /dev/sigmblockh
brw-rw----    1 root     root     254,   8 Jan  1  2000 /dev/sigmblocki
brw-rw----    1 root     root     254,   9 Jan  1  2000 /dev/sigmblockj
brw-rw----    1 root     root     254,  10 Jan  1  2000 /dev/sigmblockk
brw-rw----    1 root     root     254,  11 Jan  1  2000 /dev/sigmblockl

The highlighted line shows the major, minor numbers for this device. As you might know, the minor number is the partition number on the disk. In this case, the rootfs is stored in the 7th partition.

To understand where is the 7th partition stored in the flash disk, we need to know the partition information of the internal flash disk. This could be obtained in two different ways. Either by looking at the logs during WDTV boot (via serial port) or by issuing the following command at the terminal (via serial again).

~ # cat /proc/sigminfo 

dev:       size     offset   name        CS
sigmblk0:  0ff80000 00000000 "CS0-Device" 0
sigmblk1:  00080000 00000000 "CS0-Part1"  0
sigmblk2:  00040000 00080000 "CS0-Part2"  0
sigmblk3:  00300000 000c0000 "CS0-Part3"  0
sigmblk4:  00300000 003c0000 "CS0-Part4"  0
sigmblk5:  01000000 006c0000 "CS0-Part5"  0
sigmblk6:  00800000 016c0000 "CS0-Part6"  0
sigmblk7:  05a00000 01ec0000 "CS0-Part7"  0
sigmblk8:  05a00000 078c0000 "CS0-Part8"  0
sigmblk9:  00020000 0d2c0000 "CS0-Part9"  0
sigmblk10: 00020000 0d2e0000 "CS0-Part10" 0
sigmblk11: 00020000 0d300000 "CS0-Part11" 0

We are almost there now. This shows the starting offset (0x01ec0000) and the size (0x05a00000) of the partition. That is a 90MB partition reserved for rootfs ie., the compressed cramfs partition cannot exceed 90MB.

Read rootfs and export to different machine

This is required as we need to patch the rootfs of the installed firmware. Press 0 (via the serial port) and restart WDTV Live. This will prevent loading the firmware and will stop boot at booloader YAMON. 

Follow my inline comments in the script below.

[Gerald] Plugin an Ethernet cable connected to a valid network. 
[Gerald] net init, initializes the interface and runs DHCP client.

YAMON> net init

Ethernet driver for SMP86XX (v1.0)
(MAC 00:90:xx:xx:xx:xx)
em86xx_eth0 - full-duplex mode
em86xx_eth0 - 100 Mbit/s
em86xx_eth0 ethernet start
DHCP was successfully configured.
ipaddr:     192.168.8.104
subnetmask: 255.255.255.0
gateway:    192.168.8.2 

[Gerald] 'nflash read' command is used to read from the flash memory.
[Gerald] Based on the discovery earlier, the following command reads
[Gerald] first 0x1e00000 bytes into memory at 0x8400000.
[Gerald] syntax: nflash read flash_address memory_address size chip_select
[Gerald] chip_select (CS) is available in the partition info at /proc/sigminfo.
[Gerald] Note: Use only 0x84000000 as target. Other addresses didn't let me read
[Gerald] 0x1e00000 bytes at once.Even here, I couldn't read the complete 0x5a00000 
[Gerald] bytes at once.
YAMON> nflash read 0x01ec0000 0x84000000 0x1e00000 0 

[Gerald] Verify if this has indeed read the CRAMFS from flash. 
YAMON> dump 0x84000000 100

84000000: 45 3D CD 28 00 40 11 03 03 00 00 00 00 00 00 00  E=�(.@..........
84000010: 43 6F 6D 70 72 65 73 73 65 64 20 52 4F 4D 46 53  Compressed.ROMFS
84000020: 9C DB 0E D4 00 00 00 00 6B 26 00 00 CB 0E 00 00  .�.�....k&..�...
84000030: 43 6F 6D 70 72 65 73 73 65 64 00 00 00 00 00 00  Compressed......
84000040: ED 41 EC 03 44 01 00 64 C0 04 00 00 ED 41 EC 03  .A..D..d�....A..
84000050: EC 0B 00 64 01 19 00 00 62 69 6E 00 FF A1 EC 03  ...d....bin..�..
84000060: 09 00 00 64                                      ...d 

[Gerald] upload the file to a tftp server of your choice.
[Gerald] syntax: fwrite tftp_path memory_address size 
YAMON> fwrite tftp://192.168.8.101/sigh/sigh.dump.1 0x84000000 0x1e00000

About to binary write tftp://192.168.8.101/sigh/sigh.dump.1

Successfully transferred 0x1e00000 (10'31457280) bytes 

[Gerald] Read the next 0x1e00000 bytes from flash and upload to TFTP
YAMON> nflash read 0x03cc0000 0x84000000 0x1e00000 0

YAMON> fwrite tftp://192.168.8.101/sigh/sigh.dump.2

About to binary write tftp://192.168.8.101/sigh/sigh.dump.2

Successfully transferred 0x1e00000 (10'31457280) bytes

[Gerald] Read the next 0x1e00000 bytes from flash and upload to TFTP 
YAMON> nflash read 0x05ac0000 0x84000000 0x1e00000 0

YAMON> fwrite tftp://192.168.8.101/sigh/sigh.dump.3

About to binary write tftp://192.168.8.101/sigh/sigh.dump.3

Successfully transferred 0x1e00000 (10'31457280) bytes

YAMON> 

Modify the rootfs to your needs

At this point, we have three 30MB files and concatenating them in order, should give a complete cramfs partition. You could now extract the cramfs partition (using modified cramfsck, as mentioned in my earlier post) and make the necessary modification.

I have the following section of script added to the /init script, that gets control to my USB script as root. This lets me run scripts on startup and modify them as required, without having to reflash anything. Don't forget to update the /md5sums.txt file with your updated md5sum.

# Gerald - update - start
if [ -f /tmp/media/usb/wdtv/gerald_init ]; then
    /tmp/media/usb/wdtv/gerald_init &
# Gerald - update - end 

Use the modified mkcramfs (as mentioned in my earlier post) to repack the rootfs into a cramfs filesystem file. Split the file into three 30MB files (note: you may need to append zeros at the end to fill up to 30MB in the third part -- your cramfs filesystem is likely to be < 90MB).

Upload back to WDTV and write the flash partition appropriately

Boot into YAMON. Use 'fread' to read from tftp to memory at 0x84000000. Use 'nflash write' to write the parts into their correct flash locations starting at 0x01ec00000. Once all 3 partitions are written, just reboot. Your new rootfs should be active.

By now, you would no longer be worried about what to do if that doesn't boot. Good luck!

WDTV Live -- Firmware Hacking Series -- Part 2

Building serial cable for WDTV Live motherboard

The motherboard has a serial port that could be used for directly interacting with the device (input and output). The official firmware doesn’t expose any means of terminal access via telnet, ssh etc., So as long as you are on an official firmware there is nothing much you could do with it. When booted, the serial port provides a root shell for directly interacting with the underlying linux. However, the prime use of the serial port is that, it provides input/output right from the time the device is powered on (for eg., once known you could interact with the boot loader etc.,) -- this is the piece that helps you unbrick or play around with much lower levels on the device.

In this post, I will take you through the process of building my own serial cable for WDTV Live. First problem, this is a hardware. So you need to really build tangible things; maybe procure stuff.

Here is the pin out for the serial socket on the motherboard.

The white socket at the bottom of the picture is the serial port. As seen on the picture, the pins are +5V, RX, TX, Gnd in the same order.

Before proceeding any further, one needs to understand the differences in voltage levels on various serial ports.

PC/RS232 serial port (ones that you see at the back of your PC) uses +3V to +15V for logical 0 and -3V to -15V for logical 1.

Most electronic serial adaptors (like the USB to serial convertors etc) and many micro-controller interfaces use 0V for logical 0 and +5V for logical 1 -- normally referred as TTL levels. The MAX232 chips are typically used for RS232 to TTL level conversion so that a micro controller could read/understand the signals to/fro a PC serial port. See my earlier experiment: Serial port logic convertor.

There are some devices which operate in LVTTL (Low Voltage TTL) levels wherein 0V for logical 0 and +3.3V for logical 1 is used. These convertors seem slightly uncommon (at least I could not easily find any convertors for LVTTL levels) and unfortunately WDTV Live operates at these levels on the serial port.

So to interface WDTV Live’s serial port to our PC, we need a USB-to-LVTTL serial convertor. I actually gave a try with my normal USB to serial convertor (TTL) and I did not succeed. The interesting thing is that, I could see a lot of characters scrolling on the screen but all were junk. Basically the ones and zeros were wrongly interpreted by the computer due to the voltage level differences. There were some references on the internet to bring down the voltage by introducing some resistors, but they didn’t work either. I didn’t try to do too much experiments, as I was dealing with a higher voltage (5V) while the serial port on WDTV Live is expected to handle only 3.3V. Better not to blow up something :)

In the internet, one cable that everyone talks about, for this use is Nokia CA-42 cable. Apparently, this cable from Nokia uses 3.3V LVTTL voltage levels to interface with the mobile phones. It looks like this (USB on one end, to the computer / proprietary socket on the the other end, to the phone).

After a lot of tries at various places, I got hold of this cable from ebay India. I could only get a “used” cable but I really wanted to give this a try. To be frank, these were really hopeless tries as I wasn’t sure what was ahead; there was a long way to go and lots of things needed to go right. When I received this parcel in my office in a dirty used box, some of my colleagues really thought I was going nuts with ebay that I was willing to pay for such dirty, used, old Nokia cable :) I really couldn’t explain all this stuff to them then. I didn’t have a compatible Nokia phone to test this cable; so until I got this whole thing to work I wasn’t even sure if the cable is in working condition. I had to confirm receipt on ebay even without testing, as I can’t test this in a day and to test this I was going to cut this cable anyways :)

This web site tells you about how to use a Nokia CA 42 cable to build a serial cable. Use it as a guide to get some useful info; it is not a bible to follow. Some of the details didn’t apply when I tested; These are the important gotchas that I figured out:

• CA-42 requires a driver and is available only for Windows. I used XP.
• It does work over VM. I used VirtualBox on Ubuntu to run Win XP.
• Important catch: CA-42 draws its power from the device, not from USB port (PC). This was very difficult for me to figure out. Some sites/forums explicitly mentioned not to make use of the +5V from the serial port of the WDTV. But unless you connect the +5V from WDTV’s serial port to the appropriate pin in the CA-42 cable and power on WDTV, CA-42 will not be detected on the computer. This was the most painful step.
• I had to connect all the 4 pins of the WDTV serial port to make this whole thing work! +5V, RX, TX, Gnd.
• Serial port settings → Baud: 115200, 8N1, no-flow-control, works.

That is all with the cable. Connect the cable, power on the device. Use HyperTerminal or ZOC (I prefer this), you are all set to interact directly with the WDTV.

How and what did I do to patch my own changes permanently into the firmware? stay tuned...

WDTV Live -- Firmware Hacking Series -- Part 1

I have had WDTV Live for many years now. As with most devices at my home, the first thing I try to do is to upgrade to an unofficial firmware that gives more features and freedom. I was running it on WDLXTV (the most popular unofficial firmware for WDTV) for quite some time.

I had scripts that run on startup to do various things including

•     Installing ntfs, wifi (custom built for my usb wifi dongle) drivers
•     Choosing the wired vs wifi networks based on its availability
•     WPA2 supplicant initialization
•     FTP based NAS share mounts

etc.,

WDLXTV hadn't been updated for more than a year now and there have been recent official updates from WD (Western Digital). I needed the latest WDTV core software with new features/bug-fixes but would like to keep all the hooks and privileges I enjoy with the unofficial firmware.  So decided to build one myself based on the latest official firmware. That's where it started.

 

Firmware package layout

The firmware is packaged with at least two files in it. The normal process is to put these two files in the root folder of a pen drive and plug it in. The current firmware (after it boots) will prompt for an upgrade if the new firmware is really newer (new version > old version).

wdtvlive.ver -- txt file, with the version number of the firmware. This is what is used to decide whether an upgrade is available or not.

wdtvlive.bin -- the root filesystem of the linux. The filesystem is in cramfs format to save space. Any writeable portion in the / filesystem is mounted on tmpfs filesystem (which is RAM based, and is lost on reboot).

wdtvlive.fff -- Optional. Kernel upgrade. In most updates, we won't see this and is required only if the kernel needs an upgrade.

 

What it takes to customize

In most cases it is more than enough to customize the root filesystem and provide hooks. For eg., I have modified the firmware to execute the script /wdtv/gerald_init if it finds one, on the USB drive. I can pack any software on the USB drive and make it run automatically on startup just by changing scripts on the pen drive, without really having to repack firmware.

 

There are two problems to solve

1. What is the customization required.
2. How to take the changes into the WDTV firmware and persist it across reboots. ie., How to rebuild a firmware for upgrade.

Problem 1 is highly subjective and is a common problem and is not specific to WDTV. It is just that any new binary that you bring in, needs to built for MIPSel (the hardware architecture on WDTV). Toolchains are available from WD. I had built my own device driver earlier and it worked just fine.

Problem 2 is what I'm mainly going to talk about. There are multiple ways to accomplish this. The easier the approach, the riskier it is (naturally).

Modify firmware - the easier approach

wdtvlive.bin is the root filesystem for WDTV Live and contains the complete filesystem that is live, post boot. There are few things you need to be aware to do this:

1. wdtvlive.bin has a header and signature parts which are mainly used for error checks. So any change to the file will invalidate these. But they can be recomputed.

wdtvlive.bin : < 32 byte md5sum header >  < rootfs in cramfs format > < 16 byte signature >



2. You need a slightly modified version of cramfsck and mkcramfs to unpack/repack this cramfs image respectively. I had to download the open source cramfs package and make the changes. The main issue is with the block size being different on WDTV -- discovered by the author of WDLXTV. To know the exact changes required in cramfs for this block size change, look here. 

3. There is a /md5sum.txt file at the root. You need to make sure that you update the md5sum of any file that you add or modify into the package; if not the package is not going to load. I see these checks during the system boot.

So if you can unpack wdtvlive.bin, add your own mods and repack it as required, you have a custom root filesystem ready for upgrade (you might need to tweak wdtvlive.ver to pretend to be a higher version). All you need to do is put these 2 files in the root directory of the pen drive and boot WDTV -- you will be prompted for firmware upgrade.

That sounds pretty straight forward, except that if we mess up anywhere in this process, there is a possibility of bricking the device (at least to the users). Your player will no longer boot as there are issues with the root filesystem and you can't revert back to old firmware as it doesn't even complete boot. I have not tried with a faulty rootfs image -- so I'm not sure to what extent the protection is available before upgrade against invalid images. I doubt though. You could potentially brick the device (although not beyond repair).

I wasn't comfortable doing this, as one mistake could cost the whole device. I don't know of anyone who can fix it for me. So decided to do it myself. I thought, If I could build the tools, knowledge to unbrick a WDTV,  I could go ahead with a custom firmware upgrade. That is what I have done now and I have later realized that if I can unbrick, I could even write the customization directly into the flash memory at right locations. I have built my own serial cable for the WDTV mother board; I have also discovered the internals of the flash filesystem/partitions and their formats to patch them correctly. There is no need for WDTV's firmware update tool ie., I don't need the device to boot, to write its partitions -- the crux of unbricking.

More details to come on the internals of how I did.. stay tuned..

Watching TV over network at home

Last weekend, I managed to complete a long-pending project of mine. The idea was to setup an infrastructure at home so I could watch TV from anywhere in my home (ie., network streaming via Wifi). I missed this facility particularly when we have guests at home and we couldn't watch TV at our convenience (second TV? No, there are differences and I don't need a second one right now).

Bottom line is, I now can watch TV anywhere in my house -- on my laptop, even on my mobile!! See this lenovo touch tablet showing Oreo Ad on Vijay TV:



I had always felt that I have everything I needed to do this, but it has not been that easy before I discovered all the right things. A simple way to put it would be: connect the video/audio output of the set-top-box to a TV tuner on a comp and stream it! But a number of questions need to be answered at every stage in this setup.

1. Video Output of Set-Top-Box:
I needed a video output that is in parallel to the one that connected to my TV. I don't want to plug out and plug in every time I need to stream TV content. Fortunately, I have been using the component video of the STB for my TV due to its greater clarity, so the composite video was idling anyways. Cool.

2. Audio Output of Set-Top-Box:
I had to use a splitter here, as the stereo audio output is connected to my home theatre already.

3. Connect it to the TV tuner:
It is this stage that took me a while to figure out!! I had a TV tuner that I used and hacked long back. When it looked like everything was in place, I realized that the TV tuner only has a S-Video input and a RF antenna (cable) input. I would need a composite input socket and there is no easy conversion possible from composite to S-Video (ie., pin to pin mapping) due to the difference in the signals. I opened the TV tuner box and searched for any composite video input that probably didn't get exposed outside -- hoping I could solder them if required. This is the TV tuner motherboard:





Could not find any socket/pin for composite video. I found the TV tuner chip on the motherboard ie., Trident TVMaster. On downloading the datasheet of the chip, I could confirm that the chip in deed supports Composite video as input. The pin diagram in the datasheet had clear indication of I/O pins available for composite input. When I traced those lines from the chip on the motherboard, it ended up in the existing S-Video socket. This kind of contradicts! How could S-Video port provide a composite input?! After lots of reading, the conclusion was simple : my TV tuner uses a non-standard 9 pin S-Video socket (while a standard S-Video socket is just 4 pins -- Y, C, Gnd, Gnd). The 9-pin socket is compatible with the 4-pin S-video socket, so the remaining 5 pins could be used for other inputs. Some hardware (like my TV Tuner) use this for simplicity in its form-factor. So all I needed was a conversion cable from composite video to 9-pin S-Video -- note: this is just pin mapping, no real signal conversion. Fortunately got this simple cable from ebay. I had reached this point months back and the project stalled :D

4. TV tuner works?
I had used this TV tuner against RF cable earlier, but never used/tested with composite video; and given all these conversion thingy, I wanted to first test if the TV tuner software works directly. And, yes it did!! I just had to configure the software to use the correct input signal. But TV tuner software doesn't stream, it just renders locally.

5. Stream it from VLC:
TV tuner device creates a video device on the computer and the tuner software uses the same. My tuner creates a Video device called 'Trident Analog Video'. VLC supports playing/streaming of video devices via DirectShow. I had streamed webcam's earlier, so wasn't anything new here for me. Things that required attention was how to configure via VLC, to use the correct input (Composite) -- as this device supports multiple inputs (RF antenna, S-Video, Composite etc). VLC has 'advanced options' while streaming that lets us configure the device. Input signal had to be set as '1' for composite. Used 'avi' as the container, 'divx' as video codec, 'aac' as audio codec. I have a Asus EEEBox on my setup that is already connected to the TV for other usage. Due to its proximity to the set-top-box, I used my eeebox for this purpose too. It connects to my home network via Wifi, so it could be reached from anywhere in my house.

6. Play it from VLC:
Now that the stream is on, all I need to do is connect to it via network. I used VLC again from a Lenovo touch tablet and connected to the eeebox via Wifi. Voila! I could now watch TV on my laptop. I'm yet to figure out an app for Android that could show multimedia stream over network -- given the small screen size, I am not that keen on watching TV on mobile. And as it has to go through the whole process of capture/encode/stream/network/decode/render, there is a delay of few seconds against the video on the live TV.

Some of you might have already thought about this: the only missing thing is that I can't change channels remotely. But it is just because I don't have a network remote, yet! (I have been eye-ing on Logitech Harmony Link for sometime :D).

MyApp: Calendar Widget for Android 2.2 (Froyo)

I had been looking for a decent home screen widget for the calendar(s) on my Samsung Galaxy S (currently on Froyo). Could not find one that is free, so wrote one.

Features:
* Includes events from all your calendars on the phone (personal, corporate etc., etc).
* Support for recurring events.
* Separate section for today's events and later events.
* Auto refresh in every 2 minutes (No, it doesn't wake up the phone, if sleeping -- so don't worry about battery).
* Occupies 3 (rows) x4 (columns) cells in home screen.
* Shows up to 6 events at a time.

Note:
* Apparently Google has deprecated the Calendar ContentProvider post Froyo. So this widget may not work post Froyo (including 2.2.1, I suppose). I only have a 2.2 phone and the emulators don't support Calendar (deprecated) -- so no idea.
* Tested only on Samsung Galaxy S on Froyo.

When I get a firmware update to Gingerbread to my SGS next month (hopefully!), I shall try to port this to Gingerbread as well, if possible.

Download the .apk installer here or point your phone to this QR code to download directly



Developed and built using Eclipse Indigo on Ubuntu.

Debugging Android apps remotely via Wifi

Android has tonnes of cool features and this is just one of them. Now I don't need to search for my USB cable to start developing/debugging the apps. Also, poor Samsung, my WinXP BSODs whenever I plug out my phone from adb debugging. This was irritating and that's when I came across this useful tweak.

Requirements:
+ You need a root'ed Android phone. (I suppose, to start/stop services..)
+ You need a terminal emulator software on the phone (preferrably). Android market gives it free.
+ You should already have a working debugging setup from your PC to the phone via USB (I mean all those SDK's, tools, ADT plugin installed).

Connect your phone via Wifi into a n/w that provides access to, from your PC.

On the phone: (either via a 'terminal emulator' OR via 'remote ssh' OR via 'adb shell on USB')

$ su
# setprop service.adb.tcp.port 6666
# stop adbd
# start adbd

All set, now the adbd knows it is suppose to listen on TCP instead of USB.

Disconnect the USB if connected.

On the PC: (example on Windows)
C:\>adb devices

if this shows your device connected, run
c:\>adb disconnect

Then,
c:\>adb connect mobile_ip_address:6666
Connected to device mobile_ip_address:6666
[not to mention, you can use any port other than 6666, but use the same in both places.]

c:\>adb devices
[this should list your device.]

All set. Use Eclipse and debug just like you would on a USB connected device.

Note: This change goes off once you restart your phone. You can automate this in a number of ways.. left to you.

Seeing my apps getting deployed wirelessly onto my phone and debugging and watching those variables remotely is really cool!! :)

Custom device driver for Linksys USB Wifi on WDTV Live

There is lot of specifics involved, so before I go any further, let me clarify the title more. This post is about using the Cisco (Linksys) USB Wifi G Adaptor WUSB54GC on your WDTV Live for network connectivity. If you need to do this on your WDTV live, you would need to have sufficient understanding of Linux and shell scripting. Also, you need to have Wifi network and with appropriate network shares available via SMB or FTP and understand how you configure stuff.

WDTV Live has wireless network support, however it doesn't work on all USB Wifi network adaptors or chipsets. There is a defined list of USB Wifi adaptors that are compatible -- again this depends on which firmware you use. The stock firmware support (from western digital) might be different from the unofficial firmwares like wdlxtv. I use wdlxtv unofficial firmware but that doesn't support my linksys Wifi adaptor as well. I had bought this wifi dongle before WDTV live, so didn't want to buy another one. The result of trying to make this work on my WDTV is this post.

Apparently, the particular linksys adaptor that I'm talking about (take a look here if you want to know what I'm talking about) is based on Ralink Chipset 3070; in fact there are many other wifi dongles that are based on the same chipset. At least the unofficial firmware supports ralink 3070 chipset, however, it doesn't matter unless the device driver was built to support that particular device. ie., the device driver should have been compatible with the given vendor id and device id. This is where my device didn't work in spite of it being 3070 chipset.

I had to then setup a total cross-platform toolchain for WDTV Live by downloading the appropriate kernel source from Western Digital website. WD has provided beautiful support for building stuff for WDTV live. The source code of device driver for Ralink 3070 chipset is available from Ralink's website. I tampered the source a bit to accommodate this Linksys device too (Vendor Id: 1737, Device Id: 0077), cross-compiled the device driver for WDTV Live on my desktop Linux. Viola, my driver got loaded for that device.

This device driver was built for the kernel 'Linux WDTVLIVE 2.6.22.19-19-4'. Make sure this matches your kernel version too (this should be in line with the WDTV Live official firmware version 1.02.21).

Also, if you have this USB dongle, you should see an entry like this when you do 'lsusb' on your WDTV Live:

Bus 001 Device 003: ID 1737:0077 Linksys

INSTRUCTIONS:
1. Download this zip file that contains (rt3070sta.ko, net.mounts, RT2870STA.dat, resolv.conf, wpa_supplicant.conf).
2. Create a folder named 'wireless' in the root of a pendrive and extract the contents to that folder.
3. Move/copy net.mounts from that folder to the root of your pendrive.
4. Edit the files net.mounts, RT2870STA.dat, resolv.conf, wpa_supplicant.conf and customize with your Wifi's SSID and WPA PSK as necessary. You can use wpa_passphrase on a desktop linux to create a encoded PSK and copy that to these files.
5. More customization is required in net.mounts script as per your network to mount the right share on startup. I use ftpfs, as that seems more robust than smbfs on WDTV live.
6. Plug-in this pen drive to your WDTV live along with your Linksys Wifi dongle and reboot, your WDTV Live should join your network and mount your network share seamlessly.

I've this setup successfully working for almost 3 months now. Every time I start my WDTV live it joins the wifi network in around 30-45 seconds and the network shares are instantly available. You also don't have to worry about network interruptions, WDTV Live takes care of remounting the shares if the ftpfs breaks for some reason (note the 'xmount' in the script).

Here is the net.mounts script that I use at home (with specifics removed):

#
# Original Author: Gerald Naveen A (http://geraldnaveen.blogspot.com)
#
# WDTV Live! net.mounts to use my custom built device driver for WUSB54GC
# on WDTV Live firmwares. This script uses the wpa_supplicant to configure
# WPA based authentication.
#
# Refer : http://geraldnaveen.blogspot.com/2010/12/custom-device-driver-for-linksys-usb.html
# for more info.
#
# COPY THIS FILE TO THE ROOT FOLDER OF YOUR USB DRIVE ON WDTV LIVE!
#
# License:
# You are free to modify/distribute/use for commercial/non-commercial/
# personal applications. Any modification to this code needn't be published.
# However, any publication of this code or the derivative of this code, should
# include the original author and this license text.
#
logger -t GERALD "$0 starting..."
if [ -f /tmp/gerald ]; then
 # make sure this script doesn't run more than once. Sometimes, net.mounts
 # gets called multiple times.
 logger -t GERALD "$0 already ran." 
else 
 touch /tmp/gerald
 config_tool -c DISABLE_ETHERNET_ON_STANDBY=NO
 config_tool -c USB_POWER_OFF=NO

 # NOTE: change below line to cp from your USB wireless device
 cp /tmp/mnt/64BA-4243/wireless/* /tmp/
 cp /tmp/resolv.conf /etc/resolv.conf

 WIFI_DEV=ra0
 insmod /tmp/rt3070sta.ko
 ifconfig $WIFI_DEV up
 sleep 5 ; # let the radio come up before scanning
 #discover the current channel number
 CHANNEL_NUMBER=`iwlist $WIFI_DEV scan | grep YOUR_WIFI_SSID -A 2 | grep Channel | cut -d\( -f 2 | cut -d\  -f 2 | cut -d\) -f 1`
 logger -t GERALD "Configuring $WIFI_DEV to Channel $CHANNEL_NUMBER"
 iwconfig $WIFI_DEV channel $CHANNEL_NUMBER

 wpa_supplicant -i$WIFI_DEV -c/tmp/wpa_supplicant.conf -B
 # DHCP client has some issue and is inconsistent.
 #udhcpc -i $WIFI_DEV
 logger -t GERALD "Configuring IP Address for $WIFI_DEV"
 ifconfig $WIFI_DEV STATIC_IP_ADDRESS_OF_WDTV_LIVE netmask 255.255.255.0
 route add default gw YOUR_NW_DEFAULT_GW_IP

 sleep 15 ; # let the n/w settle before mount
 logger -t GERALD "Pinging NAS via LAN..."
 ping -c 1 -W 2 NAS_IP_ADDR_HSPEED | grep -q "bytes from"
 if [ $? -eq 0 ] ; then
  logger -t GERALD "Ping successful for NAS via LAN. Now xmounting..."
  xmount "ftp://NAS_IP_ADDR_HSPEED" NAS_1Gbps ftpfs "-o user=movies:password"
  logger -t GERALD "xmount done for NAS."
 else
  # could not reach NAS via high speed n/w.
  # try to reach via Wifi.
  logger -t GERALD "Pinging NAS via Wifi..."
  ping -c 1 -W 2 NAS_IP_ADDR | grep -q "bytes from"
  if [ $? -eq 0 ] ; then
   logger -t GERALD "Ping successful for NAS via Wifi. Now xmounting..."
   xmount "ftp://NAS_IP_ADDR" NAS_Wifi ftpfs "-o user=movies:password"
   logger -t GERALD "xmount done for NAS."
  fi
 fi 
fi ; # if already ran
logger -t GERALD "$0 complete."

In case you find issues, check /tmp/messages.txt file on your WDTV Live for the relevant log messages to troubleshoot.

Video cable for Samsung Galaxy S

Disclaimer: Although this should just apply for any other SGS, this has been tested only on SGS Vibrant, India model.

If you didn't know how this cable looks, here it is. Note the 4 terminal stereo plug (stereo plug normally has only 3 terminals).

Although one can build a cable from scratch, it is usually cheaper or same to get a read-made cable like this, instead of trying to build our own cable with just the plugs. Ok, so what is there to be done if I get the ready-made cable -- the reason is that, all cables aren't compatible with SGS video output. This post anyway has all the info to build your own cable, if you are interested.

The common confusion that prevails in the forums is that some cables work and some don't; so how do we buy the right cable. The answer is that, you don't have to worry as long as the terminals are exactly these (ie., 4 pin stereo on one end, RCA on the other end). I had bought one such cable from ebay India for just Rs. 100. You can search for "Camcorder RCA cable" and you might end up on one such item.

If you get a camcorder cable, it most likely won't work. If it doesn't work in the normal configuration (ie., Yellow to Video, Red/White to Audio), try connecting the White to Video, and Yellow/Red to Audio. This should just work.

Why?


This is how a 4-terminal stereo plug looks. Lets name the terminals as T1..T4 as in figure. The camcorder cables send T4 to Video, but the SGS sends out video on T2. On Camcorder cables, T2 is the Left/Mono Audio (White) -- thus swapping that for Video works.

T1 is the Ground. As long as the ground is not changed, you should be able to convert any cable to be compatible with SGS video out. T1 usually gets in contact with the case/body of the device, so I would think it would be always ground on any such cables for consumer devices. I also don't see any reason for a cable to insert the video terminal between the two audio terminals. Going by these rules, it is most likely that, it is either T4 (T2, T3 are audio) or T2 (T3, T4 are audio) which is the video terminal.

If you have a multimeter, just connect this cable to your phone, turn on Video out on settings and measure the DC voltage (0-20V range) on each of the RCA plugs (make sure the phone isn't playing any audio). Only the video plug will sense a voltage of ~1.5V at this point. This should identify the video plug safely without having to try out various plugs on the TV directly.

To summarize, so you can build one if required. The SGS video output pin out is:

T1: Ground
T2: Video
T3: Left/Mono
T4: Right

Getting over problems

Courtesy: Pravs World

If you can't get through the mountain, Go around it. If you can't go around it, Go over it.

If you can't go over it, sit down and ask yourself, if getting to the other side is all that important? If it is, set about digging a tunnel.

For every problem, there are many solutions. Whatever the problems in life are, you have to find ways to get over it.

MyApp: Spb Wallet to KeePass convertor

Spb Wallet is one of the best wallet applications that I've used across various mobile platforms. However, it isn't available on all mobile platforms and it isn't free either. Once I moved on to Android, I hit a road-block because Spb Wallet isn't available for Android (yet). I'm sure you would understand how painful it is to not have a wallet app, once you are used to.

KeePass is the alternate. KeePass is an open-source software for password management. To be fair, KeePass isn't as great as Spb Wallet, but does its job. Being open-source, it is available on almost all platforms including desktops.

The pain here is the conversion. I have tonnes of data on Spb Wallet that manually entering them on KeePass is a no-go. Unfortunately, and mostly intentionally, Spb Wallet doesn't export to any well known format for import into KeePass. There weren't any handy tools to convert either. Thankfully Spb Wallet had a text export and KeePass had many import options. But it isn't directly compatible, because Spb Wallet doesn't have any proper structure to the exported TXT file and the grammar is quite ambiguous. KeePass has a well defined XML structure for import (found it by doing a sample XML export from KeePass). I wrote this python script to convert the Spb Wallet TXT export file into the XML format that KeePass can understand. In reality, Spb Wallet has more "specific" fields than KeePass, so there isn't always a direct mapping. Any non-mappable field (Account Number for example) will be appended in the notes section of KeePass so no information is lost.

This script is a simple parser that understands and converts the Spb Wallet TXT export file. It maintains the internal state of parsing and learns dynamically about what the current token is -- some times even looking ahead into the file to resolve ambiguities.

This script is probably not so robust on errors. But this did the job for my Wallet export which is reasonably big.

If you find any bug on this script that you want me to fix, report here. I *MAY* fix it for you.

Here is the source:

#
# Code to Convert Spb Wallet TXT export file to KeePass XML import file.
#
# Original Author: Gerald Naveen A (http://geraldnaveen.blogspot.com)
#
# License:
# You are free to modify/distribute/use for commercial/non-commercial/
# personal applications. Any modification to this code needn't be published. 
# However, any publication of this code or the derivative of this code, should 
# include the original author and this license text.
#
import sys

def mywrite(f, str):
f.write("{0}\n".format(str));
def main():
print "\nSpb Wallet to KeePass Convertor v 1.0 by Gerald Naveen\n";
print "Report bugs at http://geraldnaveen.blogspot.com/2010/11/myapp-spb-wallet-to-keepass-convertor.html\n";
if len(sys.argv) < 3:
 print "Usage: spb_wallet_to_keepass.py <spb_txt_export_file> <keepass_xml_import_file>";
 print "\nWhere,\nspb_txt_export_file: path to the TXT file exported from Spb Wallet.";
 print "keepass_txt_import_file: path to the output XML file, that shall be imported into KeePass.";
 return;
try:
 ifile = open (sys.argv[1], 'r');
except:
 print "Could not open input file", sys.argv[1];
 return;

try:
 ofile = open (sys.argv[2], 'w');
except:
 print "Could not open output file", sys.argv[2];
 return;

FOLDER_NAME_TOKEN=1;
ENTRY_NAME_TOKEN=FOLDER_NAME_TOKEN+1;
BEFORE_NOTES_TOKEN=ENTRY_NAME_TOKEN+1;
NOTES_TOKEN=BEFORE_NOTES_TOKEN+1;
INVALID_VALUE='invalid';

next_token=ENTRY_NAME_TOKEN;
folder_name = INVALID_VALUE;
entry_name = INVALID_VALUE;
user_name = INVALID_VALUE;
password = INVALID_VALUE;
url = INVALID_VALUE;
notes = INVALID_VALUE;
valid_entry = False;

mywrite(ofile, '<?xml version="1.0" encoding="utf-8" standalone="yes"?>');
mywrite(ofile, '<pwlist>');
try:
 for line in ifile:
  line = line.strip('\r\n');
  if len(line) == 0:
   # empty line
   if valid_entry == False:
    # entry name found after folder name
    folder_name = entry_name;
    entry_name = INVALID_VALUE;
   else:
    # found the last line of the entry..dump
    mywrite(ofile, '<pwentry>');
    if folder_name != INVALID_VALUE:
     mywrite(ofile, '<group>{0}</group>'.format(folder_name));
    mywrite(ofile, '<title>{0}</title>'.format(entry_name));    
    if user_name != INVALID_VALUE:
     mywrite(ofile, '<username>{0}</username>'.format(user_name));
    if password != INVALID_VALUE:
     mywrite(ofile, '<password>{0}</password>'.format(password));
    if url != INVALID_VALUE:
     mywrite(ofile, '<url>{0}</url>'.format(url));
    if notes != INVALID_VALUE:
     notes=notes.replace('\n', '&#xD;&#xA;');
     mywrite(ofile, '<notes>{0}</notes>'.format(notes));
    mywrite(ofile, '</pwentry>');
    user_name = INVALID_VALUE;
    password = INVALID_VALUE;
    url = INVALID_VALUE;
    notes = INVALID_VALUE;
   valid_entry = False;
   next_token=ENTRY_NAME_TOKEN;
  else:
   if next_token == ENTRY_NAME_TOKEN:
    entry_name = line;
    next_token = BEFORE_NOTES_TOKEN;
   else:
    valid_entry = True;
    if next_token == BEFORE_NOTES_TOKEN:
     if line.startswith('User Name:'):
      user_name = line[len('User Name:'):].strip(' ');
     elif line.startswith('Password:'):
      password = line[len('Password:'):].strip(' ');
     elif line.startswith('Web Site:'):
      url = line[len('Web Site:'):].strip(' ');
     elif line.startswith('Notes:'):
      if notes == INVALID_VALUE:
       notes = line[len('Notes:'):].strip(' ');
      else:
       notes += '\n' + line[len('Notes:'):].strip(' ');
      next_token = NOTES_TOKEN;
     else:
      # any unknown params should go as notes.
      if notes == INVALID_VALUE:
       notes = line;
      else:
       notes += '\n' + line;
    elif next_token == NOTES_TOKEN:
     # any thing from the notes section.
     notes += '\n' + line;
except:
  print "Unknown error occured while processing the input file.";
mywrite(ofile, '</pwlist>');    
ifile.close();
ofile.close();
print "Success. Now import {0} in KeePass as KeePass XML".format(sys.argv[2]);
if __name__ == "__main__":
   main()

Download spb_wallet_to_keepass.py

Update [26-Nov-2010]:

If the script ran successfully, but the output XML file didn't work on import, it could most likely be a CRLF issue. Try this in such cases:

Lets assume your file is test.txt.

1. Open test.txt in Notepad
2. "Save as" another file, say test2.txt (Note: select Utf8 as Encoding).
3. Use test2.txt as input