Archive for the ‘apple’ Category

Parsing Appleworks and Clarisworks file formats

Thursday, January 17th, 2013

Over the past few years, when I have downtime, I sometimes like to reverse engineer abandoned file formats. It is kind of like working on a crossword puzzle with the bonus that any progress you make helps people out there who are trying to archive, index, or convert their old files.

output of hex fiend comparing two files

I’ve spent a lot of time trying to figure out file format for Appleworks and Clarisworks. My latest efforts have been to take a file, make a small change, then use Hex Fiend to compare what has changed in the binary format.

After years of off and on tinkering and documenting I finally wrote a basic parser for Appleworks and Clarisworks word processor files. I ‘believe’ this is the first free and open parser for this file format, even if it is ten years too late. I figured out a lot about the format, but it still has a long way to go. You can view my current documenting status here and download source for the parser on GitHub.

The parser so far can read:

  • document version
  • page size
  • margins
  • document content

output of parse

From what I have seen, most people trying to read Appleworks documents only really care about the document content, but I am very close to figuring out how to parse:

  • styles – (bold, italic, underline)
  • footnotes

I may not touch it again for another year, but who knows.

 Link to file format research

 Download source code at GitHub

10+ things Flash can do that HTML5 can’t

Friday, May 21st, 2010

In face of all the hype, fear, confusion related to HTML5 vs Flash I wanted to come up with a reasoned list of why I think flash is still the right choice for building many types of web applications. This is a list of things I could think of and find that highlight some areas where Flash has innovated ahead of where the HTML5 specs and implementations are at currently.

1. Touch interface support

Despite what Apple says, flash applications that were designed to work with rollovers work really well when used on a touch based interface. (see video here). Apple neglects to mention that rollovers are super common with HTML/js sites now and none of the mobile web browsers handle them particularly well. In addition, Flash ships with a full API for multi-touch that developers can use. HTML5 multi-touch support is still being worked out and is in very early stages of being implemented.

2. More Audio Codecs

Flash supports ADPCM, HE-AAC, MP3, WAV, Nellymoser, and Speex codecs. HTML5 defines an audio tag and leaves it up to the browser to decide what Codecs to support. Where does that leave you? As of this writing, Firefox will be supporting OGG and WAV, Chrome supporting OGG and MP3, IE supporting MP3 and AAC. To build a site with HTML5 you will have to choose your codecs carefully and make sure that users are using a compatible browser. As an aside, it is also possible to write pure AS3 decoders, so there is an OGG vorbis player written completely in Flash.

3. More Video Codecs

A lot like the story with audio, Flash supports multiple video codecs (On2 VP6, Sorenson Spark, and H.264). HTML5 leaves it up to the browser to support codecs so as a developer you have to know what browser your user is using and choose an appropriate codec. Encoding your video twice once for Firefox and once for IE is going to be painful. Even sites that are moving to HTML5/H.264 are using Flash as a fallback for browsers that don’t support H.264.

4. Color Correction

This doesn’t mean a whole ton to me, and might not to you, but there are people out there for which color correction is a very big deal. If you were building a workflow application for professional photographers for instance, having built in color correction could make a ton of difference.

5. Manipulate Binary Data

Actionscript 3 is capable of manipulating raw binary data. This enables you to do things like extend it to handle more file formats, network protocols, or media codecs. Javascript cannot yet handle binary data and I don’t see a proposal anywhere that would enable it to do so.

6. Peer-2-Peer

P2P is in the HTML5 spec but as of this writing isn’t implemented in any browsers. Flash supports P2P out of the box with frameworks to assist in helping you build applications that use it. P2P enables you to have end users send data to each other without a need for a central server. For example, if you had a video chat service, you wouldn’t want every user to stream their video to a central server to be connected. The load on your central server would be huge. You would instead have users that are communicating with each other connect directly to each other to transfer the video.

7. Binary Network Sockets

Like P2P, Web Sockets will take care of most of your networking needs, but it can’t handle binary data (or work over UDP). This limits the type of applications you can write. Also, as of this writing, Web Sockets is only implemented in Google Chrome.

8. Use the Webcam or Microphone

The Device APIs and Policy Working Group is supposed to be working on a recommendation for this, but there goal is to deliver a standard by Mid 2011. Once that is complete, browser makers can begin conform to this standard. If you want an application where users record photos, audio, or video – Flash is still your best choice.

9. Video Streaming

This one seems to confuse a lot of people that I talk to because they don’t realize that Flash has two mechanisms to service Video/Audio. You can host your media on a plain old web server and have it play as it downloads, or you can host it with an RTMP server and have it stream to the user. Streaming allows you to do things like bitrate adjustments (adjusting quality depending on the user’s speed) and jump around in a file. For example, when you click in the middle of a 2 hour long movie on HULU, the player jumps right to that point and starts playing. If you had this long movie hosted on a normal web server for download you would have to wait until the entire video up to that point has been downloaded.

HTML5 only defines the HTTP download type as part of the standard. There are specs out there for getting progressive streaming working with HTML5 but they aren’t all completely implemented and aren’t part of the core HTML5 standard.

10. Media Digital Rights Management

There are tons of different solutions out there for better protecting your content with Flash. I haven’t yet seen any DRM methods that work with HTML5. When I bring this up with discussions I get an earful about how DRM is evil and who cares if it isn’t supported. I tend to agree that DRM has issues and I’m not thrilled to have to use it, but try telling that to your boss who owns the content. Wether we like it or not, DRM is a requirement for many people who are serving up the content wether we object to it or not.

11. Built in Accessibility support

Developers often overlook accessibility but anyone who has worked on a site for a large corporation knows that this is a real concern since having a website that is not usable by disabled users opens them up to lawsuits. Flash supports Microsoft’s Active Accessibility (MSAA) API. As well as TimedText closed captioning for video subtitles.

Critics have rightly complained that this support only works on windows but the number of screen readers on OSX and Linux is limited and less standardized, so I see Adobe’s choice here.

I don’t see any mention of Accessibility in any of the HTML5 proposals. (let me know if it is there). I’m sure it will be trivial to write javascript that parses TimedText and syncs it with video but I would really have liked to see something about CC in the specs.

12. Massive community supported library set

This is an area where HTML5/javascript is catching up, but it is still a long way off. The flash community is massive and has tons of open source libraries for building almost anything you can imagine.

13. Full screen mode

Late Entry: 5/27/2010
HTML5 prohibits browsers from making any way to programmatically enter full screen mode. It allows browsers to provide a way to view video full screen but not other elements. With Flash a user click action can trigger full screen of any content. While some talk, legitimately, about security concerns – this feature is used successfully by many sites out there to deliver a better user experience. Full screen mode is used not just for video viewing, but for Kiosks, presentations, and many multimedia rich sites I’ve seen. I have also not heard of any nefarious sites in the wild using full screen to try to trick users. The combination of click to activate and escape to exit seem to do a pretty good job of offering an okay balance of security and utility.

There you have it

There you have it, 12 things I can think of that continue to make Flash the right choice for many projects. I’m going to keep the list titled 10 because some people will complain that #12 isn’t a feature of the platform and who makes lists of 11 things.

For the record, I’m not an HTML5 hater. I use whatever technology is best for any given job. I look forward to more standards, more openness, and more cool new tech. On those fronts HTML5 is looking more awesome by the day, but if I want to write an app now and now next year, it isn’t the right choice for everything.

Command line file organization on OSX and Linux

Friday, May 7th, 2010

Some quick CLI commands I use all the time but forget the exact syntax for.

Find all files that have been modified in the past 7 days

find . -mtime -7

Find all JPEGs that have been modified more than 30 days ago

find . -name \*.jpg -mtime +30

Move all JPEGs from the current folder (recursively) that are greater than 40k into the folder /tmp/2

mv `find ./ -name \*.jpg -size +40k` /tmp/2

Tell me info about files in the current folder

file `ls`

Tell me info about all of the mp3 files in this folder

find . -name \*.mp3 -exec file '{}' \;

Clean subversion metadata out of a directory

find . -name .svn -type d -exec rm -r '{}' \;

Use the above to make an export of a working copy by doing:

cp -R ./working_copy ./working_copy_export
find ./working_copy_export -name .svn -type d -exec rm -r '{}' \;

Any command that is deleting files will prompt you for each file. to go through them all without prompts, just run the command as root

sudo find ./working_copy_export -name .svn -type d -exec rm -r '{}' \;

The best Chrome feature: violation of Apple Human Interface

Thursday, March 25th, 2010

Google Chrome is awesome. There don’t seem to be many people out there that will argue otherwise.

One of the things I like best about it is that they seem to willing to do whatever it takes to make the best user experience out of their product. They aren’t worried about the politics or competing interests that seem to plague progress on Firefox.

In my opinion, one of the best feature of Google Chrome on OSX is that it violates the Apple Human Interface Guidelines.

Apple has very strict interface guidelines that developers by and large tend to stick to. Partially because of Interface Builder, partially because of the community, applications on OSX tend to look and behave the same. This is in contrast to Windows where Microsoft can’t seem to settle for a single Widget toolkit for their own applications. So the guidelines are great, except when they are not.

Tabs in the title bar

The most obvious violation is related to tabs in the title bar of the UI.

from the guidelines:

The only controls that belong in a title bar are the close, minimize, and zoom buttons. If a title bar is combined with a toolbar, the unified area can contain the toolbar control and the toolbar customization contextual menu (these controls are described in “Title Bar Buttons”). Do not place other controls in a title bar.

This is in some ways just a nicety, but it is a new way to think about how to group information. It separates the tabs and makes it much more clear to a user that the back, forward, and URL controls are specific to the content in that tab. This also has the added benefit of making it possible for windows to be dropped into other windows to be merged or split.

Control focus when tabbing on the keyboard

According to the guidelines, controls should only receive focus if they are text entry fields. This means that hitting tab on a form will always tab over a drop down box.

from the guidelines

In default keyboard access mode, focus moves only between fields that receive keyboard input.

Anybody that has entered an address on a website knows what a pain in the ass it is when it tabs right past the state drop down field.

Apple suggests that application designers avoid this problem by limiting popup menus to be used only when you have short lists of items.
from the guidelines:

Use a pop-up menu to present up to 12 mutually exclusive choices that the user doesn’t need to see all the time. Sometimes a pop-up menu can be a good alternative to other types of selection controls.

Good in theory, but this isn’t how popups are used. Not in applications, and especially not on the web. Chrome tabs to select boxes on OSX like it does on other platforms. Google has decided that the web is their operating system and is using the same UX design across platforms accordingly. Perhaps it is time for companies to come together and come up with some cross-OS UX behaviors.

*This is actually one of my biggest pet peeves with the apple UI guidelines. They were set in stone many years ago and very rarely change. I understand the need for consistency but I think things like tabbing to all controls on a web-page need to be addressed. Perhaps they are long overdue for a review and overhaul. Apple, after all, is notorious for violating their own guidelines.

Java keystore cert import on OSX Leopard

Wednesday, March 17th, 2010

This morning I needed to connect IDE (IntelliJ Idea) to a FishEye/Jira server that had a self signed security certificate. Since IntelliJ (or at least the Atlassian plugin) uses Java to connect to https, it fails because of the JVM’s strict security checking.

Normally when this happens, it is just a matter of installing the certificate into the JVM keystore. There is an article and code that does this for you here. This blog post even has a nice bash wrapper that will download and compile this code for you on OSX.

When I tried to do this today, I got this error every time I tried to run this tool:

java.security.UnrecoverableKeyException: Password verification failed

After a lot of digging on google. I finally found the problem.

On Java for OSX 10.6 u1 and 10.5 u6, Apple changed the default keystore password from ‘changeit’ to ‘changeme’.

Such a trivial change, but annoying because changeit had been the Sun default forever. There is a funny post on the Apple Java mailing list where an engineer at Apple apologized and just sort of said they didn’t think it would be a big deal for anyone.

This post from Matt Fleming, has some more info as well as how to change the keystore password if you decide you don’t like this change:

sudo keytool -storepasswd -new changeit -keystore /Library/Java/Home/lib/security/cacerts -storepass changeme

iPhone Hardware Accessories Roundup

Monday, January 11th, 2010

It’s been about half a year since Apple announced that they would be allowing 3rd parties to develop integrated hardware/software applications for the iPhone. Here is a roundup of what has come out so far.

wwdc keynote 2009 screenshot

In my opinion, the most exciting announcement made at the 2009 WWDC was that the 3.0 version of the iPhone operating system would support integration with 3rd party hardware. Considering the number of stupid docks out there, I fully expected there to be hundreds of new accessories for the iPhone by the end of the year.

Here we are, a little over six months out, and I decided to dig around and see just what 3rd party hardware is out there with iPhone integration goodness.

Multimedia

L5 RemoteL5 IR Dongle
Turns your iPhone into a universal remote.
link to site
WowWee Pico ProjectorWowWee Cinemin Pico Projector
Small projectors are being touted by tons of hardware manufacturers. I have yet to see anyone buy one but maybe since this one works with the iPhone it will be the first.
link to site

Lifestyle

WakeMateWakeMate
This is an armband that monitors your sleep to wake you up at exactly the right moment in your cycle. Even more compelling to me than the waking up is that it monitors your sleep and makes nice charts of it on their web site.
link to site
PedalBrainPedalBrain
This is appealing because rather than try to invent new proprietary hardware this good looking bicycle mount reads data from standard Ant+ sensors on your bike to track everything you would want to know about your riding. The mount is carbon fiber which pretty much ensures it will cost a ton but also that I will buy one.
link to site
Ant+

Commerce

iCarte RfID ReaderiCarte RfID Reader
This one doesn’t get much press, but a cheap reliable RfID reader could be just what the RfID industry needs to get their technology into smaller businesses and retail shops.
link to site
SquareSquare
Square is getting a ton of press despite their newness in the industry and the fact that their dongle looks like absolute crap and plugs into your microphone jack. Of course, their software looks tip top and the company was started by Jack Dorsey of Twitter fame. I won’t be impressed until I see one work in person or they beat out the wikipedia entry for the word “square” in google rankings.
link to site
Verifone Payware MobileVerifone Payware Mobile
A much better looking accessory from a proven player in the payment processing biz, the press is largely ignoring these guys because they are all in love with Square. What will probably kill these guys is the fact that they are a very 1.0 company and generally a pain in the ass to buy services from.
link to site

Other

Nasa Chemical SensorNasa Chemical Sensor
As far as I know, this one isn’t available for sale but it is very cool to see that those way smart folks at NASA are out hacking up their own accessories. I recently watched a documentary about the Apollo Space program and wondered if you could now replace the 1975 NASA Central Command Center with a good iPhone app.
link to site

What I’d like to see

This is a good start, but I really hope and expect to see some crazy and varied 3rd party hardware add-ons this year.

  • A real barcode scanner, you know with a lazer and not trying to make the camera do something it doesn’t want to.
  • A voltmeter
  • An oscilloscope
  • Still no Dj accessories? Despite the billion iPod mixers, you guys can’t make an auto-tune mixer or something?
  • Um, lazertag guns
  • Expensive home automation systems, so people on MTV cribs can replace all the light switches in their house with iPhones.
  • Something that plugs into my power tools. Don’t know how, but surprise me. Chainsaw hero?

MBTA Bus Tracker Widget for OSX

Tuesday, November 24th, 2009
mbta bus tracker icon This is a widget for showing you real-time data about bus arrival times for a chosen stop. Using Next-bus satellite data you can have a very accurate prediction of when your bus is coming.

Currently, the following 5 bus routes are supported

  • 39
  • 111
  • 114
  • 116
  • 117

screenshot of MBTA Bus Tracker Widget

download icon Download Version 1.0

Compiling opencv statically on OSX

Wednesday, February 4th, 2009

OpenCV is rapidly changing and there is no bundled version that comes with OSX. I’m working on some command line apps and don’t feel like explaining to my users how to install Macports so I decided I would need to statically compile it into my application. There is a OSX Framework available, but I’m not making an application bundle so that was a no-go.

Here is what needed to be done:

First off, this document on the OpenCv Wiki doesn’t seem to work anymore if you are pulling directly from subversion.

Build System – Open CV seems to have 3 types of build scripts in subversion: Make, Autotools, and CMake. It took a lot of trial and error before I realized that only CMake appears to be working, at least on my system.

You can get a copy of CMake here here.

Before you run CMake, you have to change the following files:

  • src/cxcore/CMakeLists.txt
  • src/highgui/CMakeLists.txt
  • src/cvaux/CMakeLists.txt
  • src/cv/CMakeLists.txt
  • src/ml/CMakeLists.txt
  • tests/cxts/CMakeLists.txt

Look for the add_library lines and change this:

add_library(${the_target} DYNAMIC ${lib_srcs} ${lib_hdrs})

to this:

add_library(${the_target} STATIC ${lib_srcs} ${lib_hdrs})

Once you have made that change, follow the CMake instructions here

In Your Own Project
Assuming the above worked well, you can now statically compile your application pointing to the .a files generated by your opencv build.

One thing to keep in mind, is that your application now has to be linked to OSX system frameworks. You can resolve this by adding the frameworks to your linker in your Makefile:

-framework Carbon -framework CoreFoundation -framework QuickTime -framework QuartzCore

Removing Parallels hdd lock file

Tuesday, December 16th, 2008

Parallels crashed on me this afternoon while Windows was installing patches and rebooting. When I tried to start the system up again I kept getting this error message:

Parallels Desktop is unable to access the virtual hard disk image file winxp3.hdd. The file is missing, corrupted, or used by other application.

I knew that the file wasn’t in use by any other application because Parallels was no longer running and I had rebooted my machine.

It took a bit of digging, but here is how to make it stop thinking the file is in use:

– Locate your .hdd file (usually in ~/Library/Parallels)
– Right click on the file and choose “Show Package Contents”

When the package is opened, look for a file called “DiskDescriptor.xml.lck”. This is the file it uses to determine if the disk image is in use. Toss this file in the trash and close the package. Parallels should now start up.

Upgrading vim on OSX

Sunday, September 28th, 2008

OSX ships with a respectable version of Vim, 7.0. I can’t remember why, but I had some reason where I needed to upgrade to the latest copy (currently 7.2). Here is how to do it:

In a terminal window, run the following commands:

svn co https://vim.svn.sourceforge.net/svnroot/vim/vim7
cd vim7
./configure
sudo make install

You will have to enter your password for the sudo command.

This will install a vim GUI in your Applications folder. In order to upgrade the vim in your terminal window, you have to run these commands:

sudo mv /usr/bin/vim /usr/bin/vim_original
sudo ln -s /usr/bin/vim /Applications/Vim.app/Contents/MacOS/Vim

Vim on the command line should now be updated to the latest version.

A side effect of this update is that your .vimrc file will be read from the more standard UNIX location of your home folder. To copy the OSX vimrc to your home folder so it can be read, run this command:

cp /usr/share/vim/vimrc ~/.vimrc