The Kaltura VPaaS Website – Behind the Scenes
Jess Portnoy
We’ve recently launched VPaaS [Kaltura Video as a Service].
VPaaS‘ main goal is to significantly reduce time to market, making building video powered applications and work-flows simple and flexible. It enables you to scale quickly without worrying about the “video pipeline” by abstracting all of the complexities involved in handling both Live and VOD (ingestion, transcoding, structured and time-coded metadata, delivery, distribution, analytics, search, interactivity, compliance with security and privacy policies, and more).
To achieve this goal, two major factors are key:
- API centric cloud infrastructure optimised for scaling
- Tools and documentation that are developer centric
Kaltura was always API-centric. All the applications we develop in house, be it for administrators or end-users, utilise our RESTful API to achieve their different goals. This guiding principle is the foundation of VPaaS – specialized cloud services for video API.
As we observed developers integrating with Kaltura, we realised we need a technical resources portal that will consolidate guides, SDK downloads, API console and interactive workflow recipes.
In this post, we survey the various tools we used to build the main documentation site: VPaaS.Kaltura.com, the reasons behind our choices, the technical questions we faced and how we addressed them. We hope that this will make for an interesting read and that others who wish to launch similar projects will be able to benefit from our experience.
In a follow-up post we’ll review the API Console and Interactive Workflows system at Developer.Kaltura.com.
Content Management System
The first thing we had to decide on was what to use for managing the technical, developer-centric documentation. Our original thought was to use WordPress since we’ve had a good experience with it and use it for this very blog; It’s open source, has a very large community and is very extensible. WordPress is also written in PHP, another plus as most of us Kalturians use PHP extensively.
However, like most CMSs, it comes with an administrative overhead: managing the DB and setting up redundancy and backups, managing users and their content access levels, moderating contributions, etc.
We wanted the VPaaS documentation portal to be a collaborative project that users [employees, partners, customers and community members alike] can easily contribute to. Since VPaaS aims to be a platform for developers and technical people and most contributors will be developers, DevOps engineers, or technical writers, we felt the easiest way to manage the content would be using a source control repository. That would eliminate the overhead of managing the users manually and would also provide built-in auditing and revisioning capabilities. We decided to host the VPaaS site in a public repository on GitHub, where we also get basic, web based, editing tools.
In Comes Jekyll
Our idea was to format the content as Markdown files, keep them in a GitHub repo and then generate HTML files out of them. After a quick search, I found Jekyll and was delighted to discover it was exactly what we needed!
Simply put: Jekyll looks at a root directory, iterates over the Markdown and Textile files in it and generates corresponding HTML files, ready to be served. It is written in Ruby and is ridiculously easy to set up. You install the Jekyll Gem package, edit _config.yml to reflect your site’s metadata, run one command and presto! Jekyll does its magic and produces HTML files out of your documents.
Jekyll even comes with a built-in HTTP server [WEBrick], allowing you to preview what the generated site will look like, without having to set up a full fledged web server, making testing a breeze.
Git integration
Because of how Jekyll works, integrating with our GIT repo was very easy. We set up a repo here: https://github.com/kaltura/DeveloperPortalDocs, cloned it on our nodes and added a simple git pull command to the crontab. Each time Jekyll recognizes a change in the root directory, it goes about generating a newer HTML version of the updated markdown files.
Jekyll also supports an incremental build mode. When using incremental mode, Jekyll will only re-build documents that have changed since the last run, resulting in significant performance improvements for large sites. However, this will not regenerate pages in the event layout or other include files changed.
Extending Jekyll
Like any good system, Jekyll was designed with extensibility in mind.
Jekyll has a plugin mechanism with hooks that allow you to create custom content transformations specific to your needs without having to modify the Jekyll source. There are quite a few ready made plugins you can use without writing a single line of code, see full list here.
Below are some interesting plugins we use for VPaaS:
- jekyll-git_metadata – uses the git CLI command to collect metadata about the file [contributors, last commit time, etc]. This is very useful for crediting contributors on the resulting HTML document. I made a pull request here to make it work with Jekyll 3. Viva la FOSS!
- OneBox – Onebox is a library for turning media URLs into simple HTML previews of the resource. OneBox is also written in Ruby, which makes it a perfect fit for Jekyll. We had to make some code changes to make it display Kaltura embedded videos using our native player, resulting in this pull request.
After making these changes to OneBox, all that was needed to integrate it with Jekyll was this little piece of code. - jekyll-plantuml – PlantUML allows you to generate diagrams by using a simple markup-like language. Since the VPaaS documentation contains quite a few sequence diagrams, using PlantUML means we can embed diagrams directly into the MD files w/o having to include external image files.
An important note about extending Jekyll is that it uses Liquid. This provides added flexibility and allows for dynamic decision making when parsing the Markdown files.
Files that are not in Markdown and Textile formats, are copied to the docroot as they are, which means you can also include dynamic code files in your repo and not worry about what Jekyll might do to them 🙂
Check out our Contribution Guidelines page to see the styling and notations we use.
The Search for a Good Searching Mechanism
With Jekyll, GIT repo integration and plugins in place, all that was missing was a way to perform a search across our document tree.
We considered implementing our own indexing and search infrastructure but due to simplicity and time considerations, we eventually decided to go with Swiftype. Swiftype is basically indexing and searching as a service. By embedding very little JS code in your site, you can have a full fledged search mechanism in place.
Call for Action
This concludes our brief review of the VPaaS documentation portal architecture. I hope you found it interesting and invite you to collaborate with us in making the site and documentation better. The VPaaS docs GitHub repo resides here: https://github.com/kaltura/DeveloperPortalDocs and pull requests are more than welcomed, as are comments and questions.
