Project insights: Designing a website to be served from ESP32 microcontroller

The following is a brief summary and reflections of my experience working on an open source project with Tisham Dhar (Whatnick). Many thank to Tisham for his invitation to collaborate and his generosity with time, hardware, and knowledge. My admiration and acknowledgement to the creators of Micropython and associated modules too.

The 'ESP32' and it's predecessor 'ESP8266' are a highly capable microcontrollers and at the heart of Tisham's energy monitor project. The project's goal is to produce a hardware device that logs electricity usage through a cable (eg your homeÔÇÖs power connection) via non-invasive sensors. Amongst these microcontrollersÔÇÖ attributes is the ability to communicate over WiFi. This opens the door for easy transfer of sensor data to the internet - standard internet-of-things functionality. It also allows the creation of a local network where a user can access the device with a smartphone or laptop and interact with it directly. No internet connectivity required. As the device is likely to be installed on property boundaries or remote locations where internet connectivity could be poor or non-existent this is a genuinely useful feature.

My contribution to the project was to design a website that can be served directly from the ESP32 and controls device configuration. Project bonus points if I can design a nice visual identity for the interface. Superficially the website is not complex and amounts to several forms where a user can review and input settings. Challenges arise due to the hardware and this is where the interesting journey commenced.

At the time of writing this the ESP32 (and its predecessor ESP8266) are the bees-knees of microcontrollers. An alluring balance between low cost, sophisticated features, and good performance. Value judgements are of course relative. In comparison to contemporary web delivery platforms (eg AWS servers) the ESP32 is a metaphorical gnat on the back of an elephant. Laughably ill equipped for any 'normal' dynamic web delivery solution. Computing speed and available memory are real limitations that can be quickly exceeded. A rethink of normal web development and delivery processes was in order.

Things sound dire but thankfully Tisham was already using the Arduino framework for this project. In itself not a solution, however the framework includes useful libraries to aid with programming in C and tools to upload code to the hardware for testing. With aid from existing libraries serving static web-pages over WiFi proved easy. Delivering dynamically generated pages looked possible but not necessarily the fun project I had signed up for. To avoid a potentially laborious development path, I moved my 'dynamic web-page problem' from the backend to the frontend. With the advent of frontend libraries like Reactjs and Vuejs this is pretty easy now days. Coincidently I've been experimenting with Vuejs on another project so applying it here became a nice extension to previous lessons learned. The result was a bunch of static files served from the microcontroller that were then assembled into a web application by the browser.

Dynamic content was subsequently sent and retrieved via ajax calls. Separating the templates and dynamic content made coding in the backend neater and easier too - Multiple wins and a neat solution.

The project could have been concluded successfully at this point. However one of my project motives was to learn more about microcontrollers and the opportunities they might offer a tech-curious interface/experience designer. As if on queue, Tisham disclosed a desire to try out Micropython as an operating system for his project. Micropython is a version of the Python programming language created specifically to run on microcontrollers as an operating system. Music to my ears! As a regular user, and fan of Python, the MicroPython environment was a comfort zone from which I could further explore the unique benefits and limitations of the ESP32. We could have served the previous site via micropython, however I decided to recreate it with a more conventional server architecture (if there is such a thing these days) where the server has a more active role in page construction. Several Micropython modules already exist - namely Picoweb, BTree and uTemplate - making the development process parallel to developing with Flask or Django. Frameworks that I am familiar with and have established work-flows with. Consequently site re-creation was fast and mostly trouble free. I was even able to tackle some data management concerns with my somewhat limited coding expertise.

Conclusions

Working on highly restricted hardware presented new perspectives to my existing work practices and habits. Designing, building, then rebuilding the site highlighted tangible pros and cons of various technology decisions. Much of this seems arbitrary, or with minimal real-world implications, in the context of normal web delivery.

Directly experiencing technical development and data management was surprisingly influential to my opinions of how the user interface might be best represented.

Analysing the actions of both the user and server provides an end-to-end understanding of what is going on. This process intrinsically results in an examination of each website feature from several different perspectives.

• User experience - How a user might engage with an interface, normal UX/UI stuff
• Server behaviour - Functions and processes required to deliver the website and fulfill user transactions.
• Data representation - Data traverses through database, server, web-page, and user. Each leg undertaken by a computer requires the data to be actively assigned a form. User input also needs to be guided and interpreted back into predefined structures.

With these different perspectives in mind I find a good solutions tend to bridge across the different disciplines of web design and development. Hardly surprising considering the interrelated nature of the data and processes.

As a visual designer who has moved to include coding to my skillset it is also (often painfully) apparent that visual depiction of an interface is ambiguous and open to interpretation. Documenting designs and interactions in code is (equally painfully) the opposite but highly beneficial. No detail can be left to the imagination. Relationships of all features are explicitly documented. Implied details are not possible, nothing can be left to the imagination.

With the added ÔÇÿburdenÔÇÖ of coding, interface design becomes guided towards a visual representation that is sentient of the underlying rules documented in code. Conversely server functions and data structures can be aligned to better serve the interface. Both combine to maximise resources and deliver improved outcomes for functional user interfaces.