LCT and its Tech Stack
What is LCT:
LCT is a local contact tracing application developed by Michael Corning of the Soteria Institute alongside Enduring Net’s Ser-huang Poon. It aims to help fight the spread of Covid-19 in local communities and or large organisations.
Tech stack:
LCT is a dockerized JavaScript-based web app, with a VUE.js-based UI and a Redis, Socket.io, Node.js backend.
The Imago Team’s role:
As part of the Imago team, I was charged with improving the UI, making it more responsive and better suited for mobile devices, and that is where this tale begins.
More details can be found in this great post written by my teammate Vlad
Background
At first, we began by trying to develop in Michael’s repo. However, due to the varying time zone differences causing a lot of duplicated effort, as well as the different requirements for each local version of LCT, we decided to fork the upstream repo to our Manchester GitHub account as the Master branch.
During the development, lct-manchester-integration became the de-facto master, with Michael continuously updating and refactoring his code base which was being updated through the fork in our master branch.
To get a Manchester LCT up and running, we decided to split our team into Docker, Frontend, and Backend sub-teams, with each team merging to the lct-manchester-integration
Halfway through our time on the project, Michael updated the Redis graph structure and the main operation of the warning system was completely changed, so we pulled these changes to our master and then merged master to our individual branches.
Three team members worked on merging Michael’s upstream master to our lct-manchester-integration branch and then to the rest of our branches, fixing any conflicts and build errors. Following this, the lct-manchester-ui branch got discarded and a new lct-ui2 branch was formed.
When this was finished, the codebase in our lct-manchester-integration was built successfully. So, our team thought, “it’s all done; we’re ready to ship.”
The painful realisation:
Following the successful merge of the new lct-manchester-integration to all our branches we thought we were safe, and that when the time came to merge our UI to our lct-manchester-integration branch we’d only have to worry about a few simple merge conflicts.
So, I proceeded and merged:
After a horrible decision to “Accept All” merge conflicts, it became apparent that merging lct-manchester-ui2 to lct-manchester-integration would be more challenging than we initially assumed.
In hindsight, a big part of the problem was that the UI branch/code was not completely decoupled from the backend code.
Not only did protocols change, the JSON structure was different, URIs were different, the main structure / functionally of the app changed.
I was about to sign off when everything went up in smoke. Random code I had never seen before popped up as if I had committed it, other code got deleted.
After giving up on trying to fix the merge I decided to do a forced hard reset to return the codebase to its state before that sordid merge.
I’d fixed it, or at least I thought I had, at the end of a 20-hour day, it was all pushed in, and I was off to bed.
git reset --HARD and then git push --force origin lct-manchester-integrationThe saga continued
The very next morning I received this message:
As you can probably assume and see through this conversation that I was, how should I phrase this nicely, dying inside.
All the work on the lct-manchester-ui2 branch was gone, disappeared, as if it had never existed. *cries in JavaScript*
I rang the metaphorical warning bells, blared the metaphorical sirens on our chats warning all team members to not push any new code until we recovered the lost code. I started an emergency meeting with the merge expert in our team, to try and salvage the code and save Manchester’s LCT.
All those warnings about hard resetting git had come back to haunt me.
*Quick piece of advice DO NOT do: git reset –HARD unless you are ready to deal with the consequences of possibly losing your most recent code commits. *
After an hour of frantically searching through GitHub and googling how to retrieve the code after a hard reset we finally found the code:
We decided that the best solution to salvaging our code would be to reset the HEAD of the git branch and essentially go back in time.
I then tried to cherry-pick commits, merging one by one and kept checking that the app is still working, until finally, I had recovered all the most recent UI changes.
The summary
The merge issues were caused by the complete refactoring of the backend which was too deeply integrated with the UI.
Day 1:
- The sordid merge attempt
git reset --HARDgit push --force origin lct-manchester-integration- The merge had been removed from the integration branch but also disappeared from the
lct-ui2branch. (something I learned early morning the next day)
Day 2:
- Found out about the code disappearing
- Emergency team meeting, and code freeze for the rest of the team
- Reset the HEAD of the git branch to my last UI commit before the merge attempt
- The rest of the UI team and I proceeded to fix implementing our new UI with the newly refactored frontend until we were finally able to merge
The aftermath
I’d like to say that I have fully recovered from this highly traumatic experience, however, I don’t believe I will ever forget this experience, which is a good thing I suppose.
Following those horrible two days, I immediately downloaded the git blame VS Code extension and began using it to find out the origin of code changes and commits. I now fully appreciate the power of git, and regularly make commits with smaller changes rather than large ones like I used to.
Because of this experience I now fully appreciate the collaborative nature of programming and development in general, as such, I would like to take this opportunity to thank my incredible team, especially Vlad Geana and Gabby Mannifield for staying up to ungodly hours to help me solve this mess of a merge, and for always being willing to explain new concepts and issues.