Facebook Translator Available As Open Source Code
Thrift helps keep Facebook's many moving parts working together and keeps outside developers producing new applications in the language that is best for the task.
One of the secrets of Facebook's successful operation for millions of visitors a day is an in-house code translator that lets a system developed in one language talk to a system developed in another -- typically a major stumbling block, or at least performance degrader, on many sites.
This translator, dubbed Thrift, helps keep Facebook's many moving parts working together and keeps both inside and outside developers producing new Facebook applications in the language that is best for the task, another gain for the site.
- The Best Mobile Apps are Connected
- Cross Discipline Collaboration For Hybrid Design Featuring Rhapsody and Simulink
White PapersMore >>
A year ago, Facebook announced it was making its Thrift translator open source code and establishing a project around it. Continued work on Thrift by Facebook developers and developers at firms outside of Facebook prompted the Apache Software Foundation last month to take Thrift in as an incubator project, a preliminary step often to becoming a fully fledged, Apache project.
Thrift was needed in part because Facebook has a culture of letting the engineers choose the best tool for the job, rather than restricting developers to a handful of possibilities, said Aditya Agrawal, director of engineering and co-author of paper on Thrift with technical leads Mark Slee and Marc Kwiatkowski.
Thrift doesn't translate everything from one language to another, the way a U.N. translater conveys a Russian speech into English. Rather, it is making sure the data type of one language can be translated into that of another, and it's providing a library of remote procedure calls so that one system may call for a service from another system.
"Internally, we use Thrift everywhere. It's an instant messaging service (between systems). We don't have to worry about how data will be exchanged when it's on different servers," said Agrawal.
In effect, Facebook has come up with a way to neutralize system differences, sidestepping the need to create an immense library of point to point connectors or a universal code translater -- the latter still a figment of the programmer's imagination.
Instead, Facebook has come up with "a language neutral" way to define the data an application will use, and when called on to share data, the application turns to Thrift to cast it into the form that the receiving application will recognize. In addition, it found a neutral way to implement remote procedure calls so that a call for assistance will reach a system composed in a different language and located on a different server. The Thrift Interface Definition Language is an neutral third party that lets developers label their data structures with a minimal amount of information. The Thrift code generator then reads the IDL annotation and generates the code to move the data to a different system, explained Agarwal.
"We allow interlanguage remote procedure calls, and the developer doesn't have to worry about all the coding requirements," he said in an interview.
Thrift can translate between applications written in C++, Java, PHP, Python, Perl, C#, Objective C, Smalltalk, Ruby on Rails, and several esoteric, specialized languages used in search and other missions. They include Haskell, OCaml, and Erlang.