More mobile OpenGL: fast 2D scroller

For my spectrum analysis app for the N900 I needed a fast method to render a scrolling 2D graph.

I started by rejecting these options:

• The fastest way to render is with triangle primitives.  Wikipedia says PowerVR SGX530 found in Nokia N900 can render as many as 14 million polygons per second.  SGX535 on the iPhone can render twice as much (unless the chip there is downclocked of course).

• If you need to scroll through a static image, you can simply map it to a quad and then translate the view matrix to reveal it.

These didn’t suit me, as I wanted it to be a bitmap scroller that gets updated at each frame.  So I started with a brute-force approach:

• Update the whole screen at each frame.  On the N900, you’ll need to push 1,5 MB of data each frame, and 614 Kb on the iPhone.  The scrolling is done in the bitmap itself.

This gave me around 20 fps, which wasn’t enough.  So I started to dig deeper:

• The phone supports OpenGL ES 2.0, which means it has FBOs (framebuffer objects).

If you are not familiar with the concept, FBOs allow you to create additional framebuffers (normally you have one — “the screen”), you can switch to them at any point, and all subsequent drawing routines would draw into your custom FBO instead of the screen.  The memory  for this additional framebuffer is allocated by creating a texture.  You can then switch back to the “screen” (FBO with the id of zero) and use your FBO texture like any other texture.

So I figured I could try this (for a horizontal scroller):

1. Create two FBOs.
2. Create a 1-pixel-wide texture (assuming I’m updating a single column) for the bitmap.
3. Draw our texture into FBO #1, then draw FBO #1 on screen.
4. Now we also draw FBO #1 into FBO #2, but with 1 pixel shift to the left.  This reveals a 1-pixel-wide gap on the right.
5. We go to (2) to fill the gap on the right and now switch FBOs.  So, it’s a tripple-buffered scroller (if we count the screen as one of the buffers).

If you are still with me, here is a human-readable explanation for this: I wanted only the changed part of the bitmap to be traveling to the GPU at each frame.  Textures live in the GPU memory (although I’m not that keen on the mobile architecture to ascertain that it even has dedicated memory), so presumably most of the drawing using this method is done entirely on the GPU.

The result was a not-so-pleasing 20 fps again, or maybe it was 3 fps more, I can’t remember.  And, oh, it also had a huge flickering problem due to imperfect texture mapping to screen coordinates (the screen grid is in integer pixels, but texture coordinates are in floats, remember?)

This was a bitch to implement and it didn’t pan out.  I started to think how to simplify it and this is when came the revelation of glTexSubImage2D and the brilliant texcoord solution.

• Create a screen-sized texture (in my case, next power of two for 800×480).  Put some initial bitmap into it, if needed.  Using glTexSubImage2D update a column of pixels (or more, if needed) starting at some variable offset.  Then, append the “u” texture coordinate by 1.0/width, and render.

When offset reaches the size of the texture you just continue from the start, as texture will be wrapped by OpenGL automatically, provided you set that option.  By the way, glTexSubImage2D doesn’t require the dimensions to be power of two, unlike for the size of the texture itself.

So, the goals are reached: only the modified part of the bitmap is sent to the GPU, and the shifting is done in the most natural way possible, available since OpenGL 1.0.  Resulting fps was over 30, and sometimes as high as 45 in my app, I haven’t done a clean test since I was already satisfied with the performance.

The described method will work on any platform that has basic OpenGL support.  Check out the source code of Scroller2D class for the details, and feel free to try out my Audelicious app on your N900.

Stanford’s iPhone series of lectures

Stanford published more than 15 hours of lectures (pretty thorough ones) on the iPhone development given (seemingly) by two graduate students and occasional cameos of Apple’s iPhone devs.  The set is available as a podcast on iTunes U.

On a side note, Xcode actually does have a ⌘T-like feature from Textmate, called Open Quickly (⇧⌘D), although it’s kinda dumb.

Textmate themes in XCode

There’s a handy little tool called XThemes, which feeds any of your great Texmate themes into XCode with a simple drag-and-drop.  Check out my customized eclips3.media theme:

Crumpled Paper revisited

Okay, I owe this one.  This one crumpled piece of paper has been lying around for too long.  So, lo and behold, crumpled paper, impoved: with texture coordinates and everything:

Well, not so much everything, it’s the same code with addition of a still image you see being trashed, and some lighting tweaks.  I still yet to convert it into a transition effect.  By the way, the animation is smooth, it’s just the video.

UPDATE

Another video, this one uses a capture of the current view as a texture:

Server Status widget greets Linux

I’ve been watching an introductory podcast on Mac OS X Server and got my eye on the sexy Server Status dashboard widget.  Here is a screenshot taken from the product page:

While I appreciate Apple on the desktop a lot, on the server I just want the raw horsepower and ease of maintaining, which is why Linux is my choice.  But who said we can’t have the cute stuff, too?  Meet ServerStats:

http://github.com/melfar/serverstats/tree/master

There are two directories there: first one contains a patch for the Server Status widget to use HTTP instead of a closed ObjC plugin to communicate with server.  The second one is the server-side script, which is written in Ruby with the help of Rack — a tiny footprint web server, it employs sysstat — a Linux package containing utilities such as sar and mpstat to gather CPU and network load.

More details on the github page, leave a note if that stuff works for you, and have a nice time nursing your server!

Pretty intense memory management lesson

Hey friends!  As it appears from the outside world, Cocoa memory management looks like a mess.  It has it all:

• there’s good old malloc/free and raw C pointers
• return-by-copy almost like C++ stdlib does it
• reference counting, as in smart_ptr in C++ (but, uh, not so smart — it’s manual reference counting) and all kinds of in-house libraries
• garbage collection as in Java and pretty much all scripting languages (that was recently introduced in Mac OS 10.5)

No sweat, Apple eases the burden somewhat by giving you properties which you, uh, synthesize.  That’s @property for declaration and @synthesize nameOfTheProperty in implementation (I don’t show the synthesize part for the sake of brevity).

Also, you won’t be using any of the malloc/free in Cocoa code — it’s only there for C-based APIs, and garbage collection is not available on the iphone, so I didn’t dig into it.

Like Java, Objective C has primitive types, like bool (BOOL), int or double, and object types — those that are declared as @interfaces and allocated via  [[ClassName alloc] init] or acquired in some way.  Primitive types you just want to assign as is:

int colorsInTheRainbow;
@property(assign) int colorsInTheRainbow;

/* will use assignment operator in the setter */

Structures like strings, dates or addresses you would probably like to copy (i.e., implement composition, in UML terms).  They are called value objects in Obj-C.  The way to do that is:

NSDate *paycheckDate;
@property(copy) NSDate* paycheckDate;


/* the implementation is trickier, and you'll need to release it */

Finally, for objects you want to pass by reference to modify them, or which are too heavy to copy (e.g. you would use const reference in this scenario in C++), like dynamic arrays, you use reference counting.  To up the reference count on an object, call [obj retain], to release the ref use [obj release].

This poses a question of object ownership and who is responsible to release that last reference and deallocate the object.  You might be thinking, if you ever did manual reference counting on objects, that anything returned to you from any function you are responsible to release.  That rule normally doesn’t hold in Cocoa.

The way Cocoa returns newly created objects is by allocating and then autoreleasing them.  Consider this function returning a string:

- (NSString*)employeeOfTheMonth {
 return [[[employees objectAtIndex:bestEmployeeId] copy] autorelease]; 
} 

The weirdest thing, the function ups the reference count and then says in the documentation that you shouldn’t release it.  The way autorelease works is, unlike release, it doesn’t invalidate the object if the ref count is zero.  It is practically a licence to leak memory.  The autoreleased memory gets deallocated only when the whole autorelease pool is deallocated — worst case, on application exit (that’s practically a memory leak), but Cocoa allows you to create multiple nested pools which you can release more often, e.g. when you expect a lot of autoreleases.

The first autorelease pool is created for you by the code generator, in the main func:

NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
... 
[pool release]; 

There are probably more of them created by the framework, it is documented that one is created for every iteration of the event loop.

All so called convenience methods return autoreleased objects:

[NSString stringWithUTF8String:"In Git We Trust"]

You can use this string right away without bothering to retain it, but — if you plan to keep it over longer time (e.g. longer than the function lifespan) — then you should retain it, or else current autorelease pool could get deallocated and you’ll end up with an invalid pointer.

To sum up:

• you should release any of the retain or copy attributed properties in the dealloc method of your class.  (To myself: gotta dig out why they don’t use objects for properties and release the ref in the destructor — oh, this weird Cocoa world)
• whether to retain or to copy is up to you: you’ll have to release the object either way.  You’ll see Cocoa copies objects A LOT.  And often you can either copy or retain.
• you don’t typically retain objects returned by convenience methods.  If you need to keep it, retain it or copy it (the latter is more common I guess — since you have all those @property(copy) for value objects).

On a personal note, all this memory management extravaganza feels unnecessarily complex and error prone (remember all those iphone apps that crash — most certainly caused by invalid pointers), but as a user I also appreciate the performance of Mac OS/iphone apps over those on other platforms — who knows, maybe as ugly as it seems, Cocoa memory management model is all about the speed.

Make sure to read Cocoa Memory Management Guide for a more detailed, but not so much on the hands on side, description.

Crumpled paper, take zero

In the “for fun and profit” section, I’ve been hacking around with OpenGL ES to achieve the crumpled paper effect on the iphone.  There are two ways to this: either do it programmatically — and you have to somewhat get the mechanics behind that effect in that case, or — have some 3d-modelling software to animate that for you and then export the coordinates to your app and just play them back.  It makes better sense to go with the second way, but that goes without saying that you should have the skills in whatever 3rd party software suitable for that, which I don’t.

So, I decided to give the programmatic way a shot.  That at least includes three things:

• A plane.  Which of course, you need to subdivide in an artistic way.  Right now, I went with a simple rhombical lattice.  If you take any point out of it by changing its z coordinate, it will form a 4-sided pyramid.  Of course, for a nice picture we want a more sophisticated and heterogenous structure, but it will do fine for just now.

• The crumpled effect.  In the simplest form, we’ll just explode the polygon at an increasing rate.  That’s not physically correct, since as the paper gets deformed, the distances between adjacent points should remain the same.
• The paper ball deformation.  We will slowly deform our plane to a hemisphere.  Think of the plane as a patch on a surface of a sphere of an infinite radius, which we will gradually reduce down to the screen dimensions.

So far, so good.  Here is a short video of the animation I have so far.

Next up: more artistic plane subdivision, texturing, lighting and of course, nicer animation.  If I manage to pull this off, I might as well purge this post since it’s really here only to give myself a nudge and get this thing over with. ;o)

My Xcode setup

Xcode requires a little getting used to, at least for me it did.

This is what Xcode originally looks like:

This layout makes project window look a bit useless, you can only navigate through the project files.  It actually has a concealed editor, see the horizontal splitter on the bottom of the right pane?  And yet there it is.  Move it all the way up and you will get something a bit more exciting:

That shows my current layout, it’s based on the Dusk theme with the 12 pt Monaco font.  To edit a file, locate it in the navigation pane and single click it.

If you are anything like me and also have theLongWordPhobia go to the Code Sense tab under Preferences and enable Code Completion, I opted for Immediate suggestion.  Might be helpful to customize the toolbar — as for me, I swapped the first dropdown with the one that switches between the iPhone and the simulator, and also requested smaller text size.

I hope you’ve already got rid of the welcome screen on Xcode start, so you should now be ready to create your first (or next, whatever) app.  Have fun!