Sunday, June 28, 2009

A data alignment issue -- example

=== the examples assume a 32bit compilation ===

Understanding what data alignment is and realizing the need for data alignment is a different topic by itself; I'm not going to write about it as there are lots of them around.

Issue: The first member of a struct need not me located at the starting (offset 0) of the struct instance (yes, assuming there are no virtual functions).

Unfortunately, in most cases, this happens to be true; however the point here is that it needn't be. I've personally seen this behavior recently which led me to write this (albeit on a 64bit compiler).

Consider the struct definition,
typedef struct _A {
int b;
} A;
the sizeof(A) will be 4. This is trivial. Now consider this struct,
typedef struct _A2 {
char a;
int b;
} A2;
A2 has one char in addition. Some people might expect the sizeof(A2) to be 5 -- but in reality the sizeof(A2) would be 8 due to the data alignment requirement. So where is the extra 3 bytes (called padding) gone? let's examine the offsets of the individual data members to figure out the gap.

Assuming a2 is an instance of A2,

offset of A2::a => (char*) &a2.a - (char*) &a2; // offset of a2.a from the starting of a2 => 0
offset of A2::b => (char*) &a2.b - (char*) &a2; // offset of a2.b from the starting of a2 => 4

Clearly a2.a starts from the zeroth byte, and a2.b starts at the fourth byte. The layout of the struct is as follows {a2.a|*|*|*|a2.b|a2.b|a2.b|a2.b} where * represents the padding bytes and each | represents a byte boundary.

It is important to note that, C/C++ standards do not allow the compiler to change the ordering of the struct's members in its memory representation (please let me know if someone feels this is wrong). However, if you think a while, you would realize that without any change to the ordering of the members, the padding can be moved around while still fulfilling the data alignment requirement.

For eg., the memory layout of A2 could also have been {*|a2.a|*|*|a2.b|a2.b|a2.b|a2.b} where * represents the padding bytes and each | represents a byte boundary. This is perfectly valid and easily invalidates the assumption about the address of first member of the struct -- because the offset of a2.a is now 1 instead of 0.

ok, but why would someone rely on this assumption??! Pbly not directly; it does not make sense to use a2, where a2.a is to be used. However, in nested structures, this might go unnoticed. Consider this scenario,
typedef struct _A3 {
char a;
void *ptr; // assume that by design, ptr points to A4 or A5
} A3;

typedef struct _A4 {
char c;
} A4;

typedef struct _A5 {
char c;
int n;
} A5;

void print_members(A3 *pa3)
// assume by design:in most cases pa3->ptr points to a A4 instance.
// and given that both A4 and A5 have the common first member
// it might be tempting to write a code like following.
A4* pa4 = (A4*) pa3->ptr;
printf("%c ", (char) pa3->a);
printf("%c ", (char) pa4->c); // here the code is trying to print A4::c or A5::c
printf("%d ", (int) ((A5*)pa3->ptr)->n);
The code at line 23 may or may not work as intended, based on the result of data alignment for the struct A5. This is a perfect disguise of this untrue assumption. So, Beware!!

Tuesday, June 23, 2009

TVProgramGuide - my application for tv tuners

This is about an application that I've recently developed, that can show real-time TV programme guide lines on the fly over a TV tuner video.

It is really interesting to see how subtle things can make a big difference in the way we carry out every day life. This idea struck my mind few weeks back, when I landed at accidentally while trying to find out what movies are played for the day. The idea was to integrate this info about TV programmes into the existing TV tuner application, so I can fetch them whenever I need them. When I was first thinking about this idea, I didn't really expect it to be soo useful -- today I just can't watch TV without the aid of this app.

The idea sounds interesting but it is as vague as a patent. I spent the first week thinking about the feasibilty of this application, and about how to integrate this app into the TV system. I was pretty clear that this app is going to be of no-use if I cannot provide a means to use this application wirelessly (yes, using the TV remote). If I have to come to the comp, and use the mouse to find out 'what's coming up next?', I would rather visit the website in my browser and know about it. It is as simple as having a bookmark in my browser.

But I wasn't sure if I can hook into the TV remote, and get its signals. Even if I can, I should also ensure that my hooking does not affect the normal functioning of the TV tuner application, that is already using the TV remote and reacting to the signals. As specified in my earlier post on my TV tuner, the tuner's hardware (Trident) and the TV tuner application (Honestech) seem to be from different vendors -- this gave me the hope that there should be an interface available somewhere (although unpublished) using which the existing TV tuner app is receiving the TV remote key presses. I will definitely be writing a separate post on the technical details of how and what disassembling I had to do; but for now, it is that I've managed to discover the undocumented APIs that are used internally, and the appropriate DLLs and managed to hook into them seamlessly so the TV tuner application has no idea that I'm hooked in.

This hooking was done in C/C++. The remaining task was to download the TV programmes from their website (note: oneindia does not have the list for all channels. So I had to do a generic design to support any website; with abstract classes and interfaces, this isn't a problem anyway). For any non-system programming I prefer python (if not UI related) and C# if it has an associated UI. I admire the power of the modules/class libraries that these platforms provide; awesome! .Net comes handy with Http classes (System.Net.HttpWebRequest) to handle the HTTP requests/responses for downloading the programmes. And I used RegEx (System.Text.RegularExpressions.RegEx) to parse the HTML output and extract the program schedule. With a number of choices for sharing info (remote key press) between the C/C++ hook and the C# interface, I chose the simplest one: The windows registry. I intend to post a developer post on this later, but that's the overall technology behind this app.

Important features:

1. Automatically shows the 'now playing' programme on every channel change. The 'now playing' item is picked based on the current channel no. and the current time. This window shows up for 10 seconds and auto-hides. Very useful when we glance through the channels.
2. Channel change is detected by monitoring the remote key presses including numbers, up, down, recall etc., Interesting part was that channels can be changed by a sequence of key presses eg., key 1 followed by key 2 followed by key 3 in a short interval, is not 3 different channel changes, but a single change to channel no. 123. It had to be handled differently.
3. A special mode can be entered by a special key combination (that does nothing to the tuner app) in the TV remote. In this mode, the app overlays the 'coming up next 5' programmes list over the video. This info does not auto-hide. It can be closed by the Mute key on TV remote.
4. Seamless integration, so TV tuner works just well as before.

Here is the video showing my application in action as I watch TV on my comp:

Overall, I'm very happy with this app; one of my applications that I use the most. I'm hopeful to publish this app soon, after I make some generalizations (currently channel associations are hard-coded and not all channels are supported).

Wednesday, June 10, 2009

Fronttech TV tuner

After contemplating on this for quite sometime, I finally decided to buy a TV tuner. I don't intend (or I should say don't want) to spend too much time on TV. I was fairly confident that I was not going to buy a TV for sure; and if at all I go for something that would be a TV tuner card for my existing LCD monitor.

Then, the question was whether to go for a internal TV tuner, USB TV tuner or an external TV tuner. For those who aren't aware, there are 2 types of external TV tuner -- one that drives a CRT and the other that drives a LCD. So be sure to buy the correct one incase you have a LCD monitor. However, I didn't choose this because: I don't want to keep changing the cables to the monitor everytime I need to switch between TV and comp (my monitor does not have dual input); and I might want to parallely work while watching TV (or atleast check my stock prices, email, orkut). These being the problems on external ones, I was firmly confident that the internal/USB ones will have lot more "features" than the external ones like recording, PiP (Picture in Picture), scheduler etc., and given that it was software controlled, I was hopeful to try my hands on it by writing some code for it (I had no idea about what to do with it, but just felt an opportunity).

After throwing out the external solution, I had to choose between the internal one or the USB one -- both satisfy my criteria of software control. I chose USB as it looks safer to me anytime than a PCI interface -- I'm hopeful that a heavy spike on my cable wire, would cause less damage to my comp over the USB port. It also provides the convinience of moving the TV between my comp and laptop. Also, the internal ones might have difficulty with remote due to line-of-sight issues (not sure, if they even provide a remote for internal ones).

Anyways, after all this I finally zeroed on Fronttech USB TV Tuner (to be frank, I didn't really spend time on analyzing the brand to buy; Fronttech was easily available in the nearby shops and so I just went for it). I had bought it a month back and believe me, I have no regrets today. I think Fronttech is anyway just selling it as its own brand but it is not. The hardware driver identifies itself as 'Trident Analog Video' to Windows and the TV tuner software is from Honestech. Possibly, Fronttech takes care of marketting, sales, service/support.

It is so handy, fits in your palm. This is how it looks (Photo from Fronttech).

It comes with the following key features:
1. Real-time recording (at a click of a button on remote)
2. Recording scheduler (I can just schedule a recording and forget about it; watch it later).
3. Time-shifting (as they call it -- more later)
4. Supports NTSC and PAL

Unfortunately, PiP is not available, and I later learned that it is not available in any TV tuners (at least in the ones available here in India). As it requires support from the hardware level (tuning in, two frequencies at the same time), I cannot do any magic with software to emulate PiP. Ok forget it.

Recording is awesome, with no lag in the TV playback in real-time. There are options to select the resolution of recording along with the video encoding that is to be used.

I should definitely mention about the time-shifting. An easier way to describe the time-shifting is 'pause your live TV' slogan that many digital TV ads boast about. Yes, this feature allows you to pause a live TV and continue from there. This is technically feasible (just like anyother digital TV solution) by continuously recording the TV channel in the background. One very good thing about this TV tuner is (IMHO), I can enable time-shifting only when I need it; this prevents the application from continuously thrashing my hard disk when I really don't need to pause/playback (I believe Tata Sky Plus does continuous recording).

Another "strange" thing I discovered was (at least I didn't know this earlier): The auto-scan option for channel tuning is not as smart as the ones in the real TV. ie., it does not scan the complete set of frequency (at every possible increments) and infer if a clear signal is available at that signal. Apparently, on hooking to their application, I figured out that they have a constant pre-defined set of frequencies to be tuned for a given country (I even remember a function in one of their DLLs, GetFreqencyTableForCountryName). Ok how does it matter to me? It does. They have some 360 channel frequencies pre-defined for Indian Cable TV networks, but not all of them have a clear signal or channel transmitted. The tuner application is not smart enough to discover the absence of the channel and skip bookmarking the channel. So at the end of an auto-scan you will have 360 channels bookmarked, with only around 100 having meaningful video output and scattered all over. Grrr.. There is also no way to tune a single channel. ie., you cannot say, switch to channel 1 and start tuning the band to associate a different channel to channel 1 -- because the channel number by itself defines the frequency to be used, and channel no. 1 will always be pointing to the same frequency. Thankfully, they have the option to name the channels and associate channel shifting to work only on the favored channels instead of just blindly going through all channels. I was a bit uncomfortable here, but after a month, now I'm used to it.

Important: Do remember to backup your "channel.she" file from their installation folder once you configure your TV tuner and named all those channels etc., It is useful to restore the channel list in case you need to reinstall the application or you lose it (I lost it once -- how, is a different lengthy story!!).

Except for these few glitches, over all I'm very satisfied with the quality of its work with some stunning features.

Tuesday, June 09, 2009

Focus On Yourself

Courtesy: Pravs World

Some of us waste our time waiting for people to live up to our expectations.
We are so concerned about what others are doing that we don’t do anything ourselves.

Its not so important what others are upto; compared to what you are doing.
Focus on what you do, your work; Not on others.

Thursday, June 04, 2009

MOTAS - The Mystery Of Time And Space

For those who have played and enjoyed 'Crimson Room' and 'Viridian Room' and still want to quench your thirst for such game, this should be a hurray!!

As described in their website, MOTAS is an online graphic adventure game in which the adventurer has to solve riddles and puzzles, find and use objects, escape from locked rooms, find hidden passages and be a detective and examine everything to unlock the doors of the mystery of time and space.

With 20 levels, this game has lots of fun and thrill. Unlike 'Crimson Room', this game needs a little bit of technical exposure (in multiple levels, you have to boot a computer and get some data out of it). You also need to understand 'time travel' and apply the same in the game heuristics. With little help from the online community, it should be lots of fun in completing all the levels (I believe few, very few problems in the game are very difficult to solve/guess).

Click here to play and Enjoy!