Category: Software

Hacking Firefox Mobile, Not The Good Kind Of Hacking

Hi again. Remember my first post to Planet Mozilla? I hope not. Sorry for the male enhancement pills spam. Please stop contacting me for orders.

I am in Toronto right now, spending a week with Mobile folks. It has been about 6 weeks since I started working on a mobile extension, and the list of dirty tricks we employing to get things done is growing. Some of these hacks are there because there is no alternative, and some are there because I just didn’t figure out the right way to do it.

Here is a short list:

Gestures
I reinvented them, poorly. Our extension relies heavily on the ability to support a rich set of multi touch gestures. We have a hack, similar to how it was done in Touching is Good (I know!), where we swap out the callbacks in MouseModule and GestureModule for our own. This doesn’t seem right, for a few reasons. The main one being that this doesn’t seem like a real public API, and it will be broken sooner or later. The second thing is the fact that we are relying on shaky JS to re-interpret the platform-interpreted events. So maybe there is a better way to do things. The nsIDOMSimpleGestureEvent only serves us to a limited degree, I  think there is room for a more low-level multi touch event API that is similar to the Android one. Or maybe a comprehensive high level API with more event types.
Speech
This should probably be implemented natively and have multi-platform support. Right now, in Android, we are doing Javascript <-> js-types <-> jni <-> Java. What could go wrong?! But seriously, it is kind of ugly right now, but I would like to make our JS solution prettier and useful to other extension writers who want to tap in to Android’s API and services.
Security
The Mozilla profile directory in Android is not world-readable, presumably for good reasons. Our extension’s media files need to be world-readable so that Android’s TextToSpeech service could pick them up and use them as earcons. This requires an install and uninstall hook that uses Android’s Context.getDir to get/create a subdirectory in Firefox’s top-level app directory where we could copy over those media files for system consumption. Maybe this is the best way to do it… But I can’t imagine anyone being happy about an extension writing to that directory. Sorry!
–enable-accessibility
Our extension depends on accessibility being enabled at build time. Since I am getting some face time with folks this week, it seems like a good thing to bring up.

I am sure there are other horrible horrible hacks in there. But the list above is really what I needed to confess to. Thanks for empathetic attention.

Caribou Week Who Is Paying Attention Anyway – We Need a New Maintainer

After making a non-binding resolution to report my Caribou progress on a weekly basis, I flaked. Of course. But luckily Nohemi has picked up the slack and have kept you all up to date about the libcaribou powered GNOME Shell keyboard in her more binding GSoC reports. So no more architecture diagrams are needed, you all get the idea. But if you didn’t, let me make it clear: The goal of Caribou is to make it easy to implement new on screen keyboards where you would only need to provide the view, and libcaribou will be your model and controller.

It is better to admit now than later: I will not have the bandwidth to continue to work on Caribou as my time is slowly running out. So…


What we need:

  • A maintainer.
  • A GTK module, Nohemi is working on this, but we will need similar solutions for other toolkits and fallbacks (XIM?).
  • Unit tests for the library – libcaribou is gaining features and getting complex. We need some tests here.
  • More keyboard layouts/languages.
  • More function keys for the switch scanning keyboard.
  • Finer interaction modes:
    • Modifiers, use latching and traditional key gestures with multitouch.
    • Respect AccessX settings for sticky/slow/bounce keys.
    • A more hardware-like interaction where keys will auto-repeat when held down.
    • etc.
    • … and automatically choose the right mode for the keyboard without comfusing the user.
  • Revisit the “X adapter” and maybe use something more high level.

Anyway, plenty of exciting work. Are you at the summit, please find me if any of this interests you.

I Am Speaking In Portland Next Week

I'm speaking at Open Source Bridge - June 21–24, 2011 - Portland, ORI am getting really excited about the Open Source Bridge conference in Portland next week. It feels like most Open Source meetups I have attended lately have been on the other end of the Atlantic. It will be nice to be in my element, for once. It will also be nice to meet cool localish people. I am planning to train down there and bring my bike. I hope to see you there!

I’ll be talking about inclusive design.

Caribou Week Whatever

Hi again.

Since the last time I wrote to you, dear bloggy, I have been working a lot on Caribou, made a trip to welcome Jenny back from Haiti, and crossed the continent by rail.

Let’s talk about Caribou, so much has changed!

Antler

Antler is the Keyboard UI that is bundled with Caribou. It does not have the pretension of being ready for users any time soon, it is more a sample implementation of a libcaribou keyboard,and a place for me to try stuff out and see if it would work in our platform. You could follow Nohemi‘s progress to catch up on how libcaribou is being used to power the new GNOME Shell keyboard UI. With all that said, Antler is still kinda cool. Here is a crappy video of Antler’s touch keyboard in action:

Scanning Support

I redesigned Caribou’s switch support from the ground up with the goal of simple configuration. There is still plenty of more work to do, but after looking at commercial alternatives I feel like we could do a pretty good job. Here is me typing text solely with the right shift key:

Please excuse the green/red combo

Input Method Support (bye AT-SPI!)

This is an experimental tangent, that might or might not be worth the time I spend on it. Recently Caribou master received GTK2/GTK3 input modules that perform DBus calls to the Caribou keyboard, and have it show up when and where it is needed. This has proven to be pretty tricky. I will hopefully follow up with a post about this, and some interesting hacks innovations surrounding these methods. Future work includes writing similar modules for QT3 and QT4. And having the keyboard emit key activation signals that the modules could use for inserting text instead of using XTest which feels so hackish and wrong.

Caribou Week 2.98

That diagram I drew up earlier this week? I finished implementing it. It is in the badly named ‘geometry’ branch. I decided to take a day or two and redo some of the C stuff in Vala, with the hope that it will make life easier when I start adding a bunch of classes and create the DBus bits. I regretted it when I found myself hand editing vapi files and amending GIRs generated by valac, but all in all, I am pretty happy with the choice.

If I could be sentimental for just one paragraph and say that all the hard work folks have put into GObject introspection is finally paying off. Big time. Transitioning between all these different languages at will is really amazing. I think we are entering a perfect storm where we have an extremely competitive developer story. There is really nothing quite like this. Yay.

Since I spent this week shuffling deck chairs, I don’t have any exciting new visuals. Hopefully soon.

GNOME Accessibility Via JHBuild

So a couple of weeks ago I ported Caribou to GTK+ 3 and PyGI. I also stopped testing it (and implicitly stopped supporting it) with CORBA AT-SPI. This means that if you want to download and test Caribou, or any GNOME 3.0 accessibility module for that matter, you need to jump through hoops to get all the prerequisites. There are two sane options:

  1. Install a pre-release distro like Ubuntu 11.04 or Fedora Rawhide. While the Caribou package is not necessarily up to date on these platforms, the prerequisites generally are, so you just need to clone Caribou from git.
  2. Use JHBuild. This is what all serious GNOME testers and developers use. Unfortunately Caribou is not in the current moduleset, and neither are some dependencies. Fortunately, I wrote a small moduleset file that takes the shortest path to having a working version of Caribou for your testing needs. It also has Orca, and other modules will be added in the future. This should probably go into upstream jhbuild eventually.

Once you have installed JHBuild, use the configuration below in your .jhbuildrc, or create .jhbuildrc.a11y, and use jhbuild -f if you have a working jhbuild environment you don’t want to mess with.

build_policy = 'updated'
moduleset = 'http://people.gnome.org/~eitani/a11y/gnome-a11y-3.0.modules'
modules = ['meta-gnome-a11y']
checkoutroot = os.path.expanduser('~/gnome-a11y/source')
prefix = os.path.expanduser('~/gnome-a11y/install')
skip = ['gudev']

Next, run jhbuild bootstrap to install build tools, and install some other development libraries that are worth skipping in jhbuild (for now it’s just gudev).

In Fedora:
yum install libgudev1-devel

In Debian/Ubuntu:
apt-get install libgudev-1.0-dev

Run jhbuild build, and watch things compile successfully.

Now, the hairy part about GNOME accessibility is the fact that there is an AT-SPI registry that needs to start with the session. Optimally the registry runs on a dedicated accessibility D-Bus bus. First you need to make sure you installed and enabled at-spi2 through your distro. On Ubuntu you might want to install it via PPA, on Fedora 14 it already is in the repositories, search for at-spi2. Next you need to enable desktop accessibility and D-Bus support:

gconftool-2 --set /desktop/gnome/interface/at-spi-corba --type bool false
gconftool-2 --set /desktop/gnome/interface/at-spi-dbus --type bool true
gconftool-2 --set /desktop/gnome/interface/accessibility --type bool true

There are a lot of tricks to get the newly built registry to run in your session, but the easiest way is also the most hackish, simply edit /usr/share/dbus-1/services/org.a11y.atspi.Registry.service and substitute /usr with the prefix to your newly installed registry, for example /home/eitan/gnome-a11y/install. Log out and back in again, and you should be using the latest AT-SPI2 registry.

If everything worked, running jhbuild run caribou should bring you the latest version of Caribou.

Good luck!

Biannual Update

It’s been almost six months since my last post. So why not just recount all the major things that happened last year. I have been extremely busy, I have been working full time for Collabora, and at the same time managed to always have some project or other deadline I was racing towards. This is probably the first time I paused to think about the wild goose chase that was 2010. Here goes:

Collabora

I have been drafting Telepathy specs, improving Gabble, drafting more specs, and generally learning a lot. Some (potential) user-visible features include bomb-proof XMPP invisibility, interactive authentication, power saving, communication policy (contact blocking), improved codec-conflict user messages in VoIP and video sessions. And more that I can’t remember right now. Of course this is on top of the client work that is not public yet. It’s been a great experience, working with energized smart people like Simon, Will and Sjoerd is fantastic. I would love to head to FOSDEM this year and see everyone, but I will be turning 30, so probably not.

GNOME Accessibility Team

Early last year Willie Walker’s Orca team in Oracle was finally liquidated and we were not sure where things were headed. I was working on putting a hackfest together, giving a talk, organizing a booth, and generally having a presence in CSUN. This was the first a11y meetup, and I spent time preparing a presentation, dealing with sponsorship and travel, getting resources for the hackfest and booth. Besides saying goodbye to Will at CSUN, it was also a disengagement for me, as my job and other projects were keeping me elsewhere.

Since that hackfest there has been another one in Seville, Alejandro and Joanie have been leading the GNOME a11y team like nobody’s business. There is even a weekly meeting that I never attend on a regular basis. Things are looking good, it is really humbling. Joanie started doing this work pretty much at the same time I got involved in a11y, and she is still plugging away. Somebody please hire her so she could do this full time.

I also gave a talk at GUADEC about Universal Design. I learned a lot preparing it, and I hope to continue polishing it.

Getting Closer

At some point this year, I found the time to develop an iPhone application, a GIS Django backend and a web-based map authoring tool! The idea of the app is basically aural augmented reality. For example you could walk around in a city neighborhood and listen to past events and stories of different places. The main difference from an audio tour is that you are being led by sound as it gets louder when you approach as opposed to visible landmarks. Of course the technology is the trivial part, the real magic is in the well-produced audio. Jenny Asarnow has been doing that, and created fantastic soundscapes with stories and music. Full discloser, she is also my sweetheart. Getting Closer was showcased in two events: Megapolis in Baltimore and Third Coast in Chicago. We hope to collaborate with more audio producers and find new and exciting uses for this. As for my end, it was fun to step out of my niche Desktop programming and do some mainstream stuff, like iOS, Django and HTML/JS, just to know I could.

Causing A Nuisance

I joined other young Jewish activists in New Orleans this fall where we successfully agitated the Jewish American community into conversation by interrupting the Jewish General Assembly, the largest annual event surrounding Jewish philanthropy. Disrupting the Israeli prime minister’s keynote was really just an added bonus. This is one of the most important things I have been part of this year, and I am completely flattened by the amount of positive responses we have gotten for this. I wrote an essay about the experience that I hope it will be published soon. I am looking forward to continuing to be part of this conversation.

Caribou

After things settled down, and I caught back up on work, I started giving Caribou some desperately needed attention. I asked that it be pulled from the GNOME 2.32 module set in the last cycle because I just never had time to really get it ready for release. No excuses! I spent the holiday slowness porting Caribou to GTK3 and GObject introspection. Let me tell you, it was not trivial! I ended up rewriting whole chunks of Caribou, which was not all that bad. Since taking over maintainership I have done little in doing major changes, and accepted some large patches that I never should have, so it was cleanup time. This whole ordeal did not introduce any user-visible changes besides a new prefs window and stability. Um, I guess that is something. Anyway, I will be merging it to master today. That bad news (and this is what sucks about GNOME 3.0), is that other contributors and testers will need to do a jhbuild dance before being able to run any of this. I have a moduleset I will share with the world that does the minimum required stuff for a development a11y stack, with the new AT-SPI2, Orca, Accerciser and Caribou. There are some amazing artists in the GNOME community, anyone fancy on designing an icon for Caribou?

That’s Not It!

I am sure I forgot stuff.

My Perfect Data Backup Solution

This might already exist, and I might just be uninformed. When I look for a backup tool that will do what I need, usually I get a scoff in the form of “rsync, dummy”. My computing equipment typically consists of a laptop that I use everywhere, and sometimes even at home, a headless computer at home that I use mostly for music and movies (with a projector), and several external hard drives with varying capacities.

These are my requirements for an awesome backup solution:

  • When I connect to my home wireless network I want it to automatically start syncing and backing up data to the headless computer.
  • I don’t want it to saturate the network or be too taxing on disk I/O, so that regular computing tasks could be resumed unhindered.
  • I want it to be resumable, so if I leave the house while it was syncing some huge file, it will just continue where it left off when I get back home.
  • A visual status indication of whether a sync is taking place. A way to pause it.
  • I want it to have 3 different kinds of backup modes for 3 different types of data:
    • Home folder
    Snapshot based backups, that allows me to easily roll back and view previous revisions of my home directory.
    Media
    Accumulative backup. See what new music, photos or videos I have on my laptop, and copy them to the backup storage. This mode never deletes media that existed in previous syncs. I should be able to download or create new media, and have it stored on the high capacity storage at home, and only carry with me the media that I am currently consuming (I really don’t want to clutter my laptop disk with The Godfather trilogy, but I might want to take it with me on a flight).
    Virtual machines
    I usually have a small collection of virtual machines on my laptop. They are by far the largest single type of data I have on my machine. Since they support virtual disk snapshots natively, I don’t really care for revisioning like the home folder, virtual machine disks could clobber older versions of themselves on the backup storage.

There are countless backup solutions (read: rsync wrappers) out there. Do any of them answer those needs in a user-friendly way?

It’s By Design

Off to GUADEC tomorrow! I feel relatively prepared, I shaved and packed 4 US-Europe plug adapters. If you forgot yours, don’t buy one, you could borrow.

Wheelchair access toilet stall with paper way high up.

I also prepared my talk. You should come, it’s on Wednesday at 11:15 AM in the Seville room. It’s going to be extremely nontechnical, but I hope you could follow. I don’t think a single acronym will be mentioned, although I don’t hold me to that. We will be discussing the software development process, design, things we take for granted every day, and door knobs.

New Laptop: Thinkpad X301

Do you know how sometimes there is something old at the back of your fridge that you are scared to take out because it probably smells super bad? And you know how it doesn’t get any better with time as you think “if it wasn’t nasty last month, it has to be nasty now!”.

That is sometimes how I feel about my blog, if last month I didn’t blog in a long time, today I didn’t blog in a really really long time! So I am probably officially not a blogger any more.

I still owe some obligatory posts, specifically about my day job, and about the fact that  and giving a prez there.

But now let’s talk about my new toy. Did I need it? Probably not. My T400 is a portable workstation that gives me everything I need. But every 2 years or so I decide to splurge on a new machine, and I am happy to say that I have become increasingly satisfied with my choices each time. My T400 refuses to die or become obsolete, it’s black unimpressive looks doesn’t allow it to get stolen. But it’s been 2 years!

I have been looking at the Thinkpad X series for a while, and debating whether the X201 was ergonomically large enough for full days of work. The X300/X301, priced north of 3k, always seemed obscenely expensive, but I was curious what made it so (is it the whale penis leather exterior?). It’s been lauded as Lenovo’s answer to the Macbook Air, but I have seen those in reality, they looked paper thin. The X301 does not look as miraculous, not even in the sales photos.

One day, while reading the interblogs, I came across a $1,700 discount for the X301. I impulsively clicked “checkout” (after clicking “add 2 year warranty” and “add to basket”). Then came my favorite part about ordering a Thinkpad: the weeks long wait for them to assemble it, ship it, get through customs, and hoping the UPS does not lose it in the back of a truck in rural Wisconsin. After this period, you are sure that the UPS guy is going to deliver pure joy to your doorstep, as this photo suggests:

It’s a pretty nice machine, everything I liked about my T400 in a reduced package. Large enough to work on, but extremely light. Is it worth the $3,000 price tag? No. $1,300? Yes.

My favorite feature is not the 128 GB solid state drive, it’s the fact that the earphone jack is on the side, and not in front like in the T400. The front jack just didn’t work for me, it was super awkward.

My biggest disappointed has been the fingerprint reader. It does not work on Linux, what’s up with that? Are we in 1995? I really looked forward to unlocking my GNOME key ring with my middle finger, that would be so cool.

OK, enough with this guilty materialistic blogging. Next post will either be about my upcoming GUADEC talk about universal design, my fun happy times working on Telepathy and friends, or Israel’s rapid decline from nationalist chauvinism to overt fascism.