May 8, 2015
TABLE OF CONTENTS
In a previous blog post, Refactor Impossible (Part 1), I wrote about some of the organizational challenges developers face when a code base needs to be refactored. The key idea was that feature deadlines and lack of immediate priority from business partners shouldn’t stop developers from working towards a quality code base. As developers, quality code is our responsibility, and if we don’t find a way to accomplish it, no one else will. At the end of the article, I mentioned a recent project we worked on that really put some of these ideals to the test.
Just like most business teams, the client wasn’t interested in a big refactor for the sake of code quality, especially at that stage of development. They were, however, really interested in performance improvements because that was one of the biggest pain points for their early pilot users. If we could tie a refactor to major value for the business (e.g. massively improved performance) we’d probably find the wiggle room we needed to put the code base back on the right path.
We then spent a couple hours profiling the app and quickly figured out the bottleneck was WebSQL. The app worked by syncing large amounts of data from the company’s servers to a WebSQL database in the browser and then every page would pepper the WebSQL database with queries to load and save the data it needed. Each query was pretty slow by browser standards (~300-500ms) and there was no way to cache the queries because every new page load would wipe out any variables kept in memory. The browser’s sessionStorage and localStorage weren’t options because the size of the query results easily exceeded their 5mb quotas.
It was evident that the app would be better off as a Single Page App (SPA). If it were a SPA, we’d be able to load data into memory at app startup and increase performance by offloading database calls from WebSQL to memory. A SPA would also give us maintainable view rendering (instead of jQuery), and a module system for better code organization. Unfortunately, it was not a SPA, and converting it to one would have required re-writing the entire code base.
Still the promise of single page-ifying it was too big to give up, so we put our heads together and came up with a plan. What if instead of converting the UI into a SPA, we wrapped it in one? We quickly threw together a concept app using React and React Router that would examine the browser url, and render the appropriate page inside a full sized iframe. The app would then monitor the load event of the iframe and update the browser url via pushstate every time the user navigated to a new page. The browsing experience for the user stayed exactly the same as before, with unique urls, forward, back and refresh support, while developers instantly got a SPA architecture to work within. The benefits were huge.
With the new SPA architecture wrapping the original app, we were able to move application data management into an in-memory database using LokiJS. The results were phenomenal, with a 100x to 200x performance improvement across the board.
Even though we felt the approach was a bit of a long shot, the architecture proved to be solid. It even handled querystrings and url hashes without much effort. It’s a pretty cool pattern. If you need to refactor a traditional web app into a SPA or are just curious how the code works, the proof of concept is shared below.
Check it out, let us know what you think, or fork it and show us how it can be better.
At the end of lunch with a mentee, I used the items on our table to express the fundamental concepts of Kubernetes. Sometime after explaining the purpose of the Kubernetes scheduler, she asked a question I spent the next several weeks thinking about.
API design is crucial, giving structure to application interaction. Given cross-functional teams and applications, development time is reduced with a clear, intuitive way to access data. API development often follows two approaches: REST and GraphQL.
As of June 2018, the state of California passed a new privacy law that could lead to more consequences for US-based companies than the European Union’s General Data Protection Regulation (GDPR). Here's what you need to know and how to be compliant.
Before your data scientists wring value out of your reams of data, it has to be accessible and, on some basic level, coherently arranged. To harness all that brainpower, you need to keep the data wrangling to a minimum. Enter the data lake.