Re-Creating Sci-Fi Computer User Interfaces in Real Life

Read the article via this LINK.

Story

One of the perks of being passionate about HCI and UIs is that I see UIs everywhere, even when they are not real. Whenever I watch a movie, TV show, or play a video game, I almost immediately noticed the computers being used and their UIs, even more so with the Sci-Fi genre. So back in around 2020 I had this idea of investigating how well or practical will these UIs be if they existed in real life. This had led to a really cool paper I published with my postdoc supervisor.

About 10 days before Summer 2023 I was asked to cover for an upper-division course. With only a bit more than a week to prepare I said the only way I could do it was to reuse the materials I had in a similiar level and the topics I was familiar with. What I had was the materials I used for teaching a postgraduate course on emerging technologies (e.g., deformable displays, wearables, virtual reality). And so I proposed to use it as a framework but put more emphasis on the UI topics. Another consideration was that the course was supposed to be covering topics in computer graphics, so I narrowed the UIs down to something that had graphics components in them.

The result was a really interesting course where students worked as groups on a multi-phase project, each in a different format: a written proposal, a low-fidelity prototype, a presentation with demo, and a written report on their refined prototype. And what the projects were about, as you might have already guessed, were to recreate a Sci-Fi Computer User Interface as selected by the groups.

At the end of the course I decided to write something about it so not only I can better teach it next time, but also for someone from another institution who wants to teach something like that. I eventually sent this article to Interactions as its focus is related to what I do (I have a subscription of it), and the nature of this article fits it quite well (and thus a better reach). I’m glad that they accepted my submission and published within half a year.

Reflection

This is one of my favourite courses. Not only did I have fun teaching it, but my students also had fun building they own stuff in groups. Some of them ended up putting their work on their portfolios. One told me that “that project was my favourite thing that I have worked on throughout my undergraduate degree”.

There are definitely several things I could improve upon, but I think the general structure was good and gave my students a feel of research projects (this course was based on a postgraduate course where students had to read papers and write something submittable to a conference).

The struggle I had was how open the projects should be. On one hand I want my students to freely choose any UIs to recreate, but some would be too impractical to implement (e.g., a HUD with perfect object recognition capabilities) or I just don’t have the expertise in guiding and accessing them (e.g., AI); on the hand I also don’t want everyone ended up recreating the same UI. I’m working on reducing the scope and selecting a number of Sci-Fi UIs for starters to determine what is a feasible UI to recreate.

In the long run I also want it to be offered in a more regular basis, perhaps not as a degree-fulfilling course, but more of a special project course often offered to upper-level students who want to work on something more long-term and open-ended.

Recommended citation: Victor Cheung. 2025. Re-Creating Sci-Fi Computer User Interfaces in Real Life. interactions 32, 3 (May - June 2025), 8–9. https://doi.org/10.1145/3721088
Download Paper