Virtual Reality

TASK 3

VR Limitations

Since I do not have a VR headset, and buying one would be an inconvenience, I am limited to only testing the game at college where I have access to a VR headset. This won't affect me too much however, as I don't need to use a headset when making my UI, only for testing it in Unreal. However for Harrison and Kai, who are using Unreal much more closely, they need VR headsets to test and try new ideas. Kai has a VR headset though, so we ask him to test the game after updates.

Portfolio Development and Design

I developed my portfolio in Unreal Engine to both show off my experience in the engine, and be able to easily change the examples of my work within the portfolio itself. Employers can then see my skills in action through the portfolio, instead of a stock template on a website building site that I used because I couldn't be bothered to make it myself. This also shows my commitment and perseverance in the UI field, something employers would want.

The Main Tab System

This was made by using a Scroll Box, and setting the widget to scroll into view to be a section (e.g. About, Examples, etc.). A handy action for this is the Scroll Box's "Scroll Widget Into View", in which I use a "Widget To Find" variable as the Widget To Find input. I then set this "Widget To Find" variable each time I click the up/down arrows, or a tab to be the correct section that needs to be scrolled in.

This is all done in a function that is called each time the tabs or the up and down arrows are clicked to save it being on Tick, which would cause unnecessary load and slow down. I also created two Custom Events to handle the up and down arrow buttons in case I wanted to reuse them later.

The background colour change was finicky to make, because UE4 doesn't allow multiple different components (even if they're the same type!) to be connected to one variable set. This forced me to make a chain of sets for each button that I wanted to change the text of for each change in colour in the background. For the background colour itself, I made a struct that held the three colours and changed them depending on the Current Page variable not being equal to the Page Number variable. The Forward Custom Event is from my custom Blueprint Animation events from UMG's timeline using a Repeater to allow me to change the colours on the fly, which I used to set as the background colours structs. I explained how this works at the end of my Project 1 UI.

Making each section

Each section is its own Widget that's put into the main Widget with the tab system. Most of them are just comprised of simple text and buttons that link to external web pages through the "Launch URL" action, but one (the Examples page) uses its own tab system (the same as the main one actually but without the up and down buttons, and it's horizontal instead of vertical) which showcases examples of custom Widgets I've made.

In order to show the functioning widgets in the portfolio, I had to migrate them from the projects they were made in into the portfolio project so I didn't have to remake them. The one in the screenshot is the style I used for my Project 1 UI. The other styles are Minimalist, Rounded Corners (using a material), and animated Blur Effects. These show off my knowledge of importing and using images in the brushes, as well as animations and materials at the same time. To show off my Blueprint Animation Macro, the Rounded Corners style has multiple different colours it changes to, which you can't do in normal animations unless you make lots of images, which isn't ideal as it's more load on the computer.

Researching how to export to HTML

Naturally, I want to showcase my portfolio in the easiest way possible, and making someone download a potentially harmful executable isn't the best way to do that, so instead I want to export it as HTML and host it on a website. Unreal Engine's HTML export is currently still in development, but it can be used, just with varying results.

After reading through the instructions on building the engine from source to make the HTML export available, I've come to the conclusion that unless it gets incorporated into the engine soon without me building it, I won't be able to use it, as I've never built anything from source before, and have no idea what I'd be doing with it. So instead, I will record a video of it, put that on the website in place of where the HTML export would be, and provide a link to download it if viewers want to. If not then they still have the video to watch instead, which should still provide all the information they need from it.

UE4 Widget Troubles

When trying to attach a 3D widget to the left motion controller, I encountered a bug where it would completely break the movement of the player, not letting them teleport anywhere apart from where they are. It would also put them in the air, which was very confusing, as creating a widget shouldn't change anything. I then deleted the widget and removed all the references of it, which should fix the problem, but it didn't. I then restarted Unreal Engine and it fixed itself. Why this happens I'm not sure, but I'm assuming it's because the widget wasn't cleared from the cache when deleted, which has happened to me before with something similar, but that still doesn't explain why it broke the player's movement in the first place.

After restarting UE4 with the widget still in place, I found I could move and interact with things again. So it seems to be more complicated than "it staying in the cache", as it works even with the widget after restarting, so it's not the widget blocking anything. The answer to why it didn't work before restarting however is extremely confusing and hard to find, but if a restart is needed after every change or addition, then it would be a useful and necessary answer to have.

Source Control

In order for the three of us to edit and change the project when we're at home, we needed to set up source control. We couldn't do this at college however, as the computers don't have the right permissions to allow us to set it up. We had to do this at home, therefore, which was fine, it just required a bit more work as we couldn't be together to do it.

GitHub was the decided platform for the source control, as it's free and relatively simple. It lets everyone clone the repository and commit their own changes to the master (or another branch) which everyone else can then get.

Making the widget interactable

After creating the 3D widget, I realised I couldn't interact with it. I know that if the rendering space of the Widget component is World Space, then it doesn't interact with the mouse (even in a normal setting, not in VR). In Screen Space however, it works like a normal UI but just stays in the same position as the actor it's a component of. Using Screen Space in VR doesn't show the UI, as the VR camera can't render screen space items and UI widgets, hence the need for 3D widgets, because a UI stuck on your face can be very disorienting (unless it's a really well done HUD). This means I have to use World Space, and find a way to let the right hand point to the UI to interact with it.

The UE4 VR template has Widget Interaction components attached to both controllers already, and these let them interact with 3D widgets in the world. However, for whatever reason, after following a tutorial, I couldn't get them to work. I enabled the "Draw Debug" boolean to see the pointer's line, but couldn't get the widget to be interactable still.

After reading a Forum post by the user "le.dr262", I changed the Trace Channel of the Widget Interaction component on the VR controllers to Pawn instead of UI, and thus the widget now blocks the trace from the interaction component! In the Widget actor, I added a Clicked event on the button to quit the game to test if it works, and it does. This is because I took the default TriggerRight Input Action from the template (which is the right trigger of the right controller) and added "Press Pointer Key" and "Release Pointer Key" functions to its Pressed and Released states for the right controller's Widget Interaction component. This essentially simulates the user-chosen key press (most cases would be left mouse button) on the point at which the trigger is pressed down, acting the same as a mouse click. This is what I will use for interacting with the buttons in the game's menus.

The 3D Widget actor before changing the Trace Channel to Pawn. It would only interact at the very centre of the actor and it took us a long time to figure out to change the Trace Channel.

Pressing the Left Mouse Button when the right trigger is pressed.

Setting the Trace Channel to "Pawn" instead of "UI".

Creating a visual timer

To make the game more competitive, every level will be timed, but won't have a time limit. This warrants the need for a visual timer that the player can look at to see how long they've spent in the level.

I made a Custom Event called EverySecondEvent of who's delegate goes into a Set Timer By Event which loops every second. That timer is run from Event Construct to save memory usage. In the Custom Event, I increment a variable called "SecondsTime", which then goes into a Branch to check if that same variable is >= 10. This is here to add a 0 on the left of the seconds text if it's in single digits to make it look like a timer (so it'll be 03 instead of 3 for example). If SecondsTime is >= then it doesn't show the 0 (as the seconds will be in double digits so it isn't needed), if not however, then it will show a 0. The 0 is shown through an append to a string with 0 just being another part of the append. When the seconds counter is in double digits, a similar append is used, just without the 0. It also uses another Branch to then check if the seconds is >= 59, incrementing the MinuteTime variables and setting the SecondsTime to -1 if it is. SecondsTime is set to -1 because it starts one higher than what it is set at for some reason.

Designing the UI theme

Since the game takes place in a fever dream from a boy in a coma, the UI should resemble a hospital as much as possible. He has a fear of lifts, so that will also play into the UI as a form of semi-nightmare. The first design I created was based on the NHS styling in the form of a building with the main menu options (Play, Settings, Quit, etc.) and sub menus in those options made from direction signs that mimic the NHS logo.

However, after looking at the bigger picture of this, I realised that I was creating it for a normal 2D viewport instead of a VR viewport. After realising this, I didn't know how to incorporate it into virtual reality, as a building that lights up would be odd if it wasn't to scale. So instead I decided to use an old CRT monitor and display the UI on that so it's not only diegetic, but also keeping with the hospital theme, as hospitals always use old and out of date technology.

Windows 2000

Hospitals use Windows 2000, so to get the authentic hospital experience, designing my UI to look like and resemble Windows 2000 is needed. This isn't too hard though, as Windows was very basic back then. All that is needed for a button is two images, a normal idle image, and a pressed image (there weren't hover effects back then). I didn't want to completely copy Windows 2000 though, so I decided to use the same general palette as Windows 2000, but not use the same colours for the same things. So although it has a different colour, the feel of Windows is still there through the colour relationships and their lightness in comparison to each other.

Recreating the Windows 2000 button to get a feel for how it should look.

To make the image pixel perfect, I set the "Filter" to "Nearest" which disabled antialiasing on it. I also set the Compression Settings to UserInterface2D so they can be used easily in Widgets.

I made a child of UE4's default Button class to have an easy button to test the images with. I changed the Padding on the button to mimic how Windows 2000's buttons look (the contents of the button (text normally) moves down and to the right a bit when pressed to help sell the 3D bevel effect that's on the button itself).

Colour Palette

To get an authentic experience with the UI, the colours must be on-point. Therefore I have conducted research into the capabilities of early Windows machines to limit myself to their colour palettes. By using this list of software palettes, I found that Windows and IBM OS/2 used 16-bit colours, which made specific colours grey because of the limitations. I will therefore only be using this palette. I have two approaches with this, one is to make the UI with this palette as the target and therefore not have any conversion between different palettes, or make the UI with another palette and convert it to this palette, which is what images did. However, the Windows OS itself was designed for that colour palette, so I will take the former approach and make sure to consistently limit myself in that regard.

Desktop Icons

Windows 2000 is the OS that started trying to incorporate 3D-styled icons for programs and files. This is seen most prominently in the Recycle Bin icon:

When making my bin icon, I wanted it to be British in some way to resemble the location, so chose a wheelie bin, as those are very British. Something I realised when making the bin is that either the Windows 2000 icons are not actually from Windows 2000, or the colour palette I found is incorrect. Considering I found the palette from Wikipedia, this may be the case. Anyway, from using that colour palette for the bin, it looks very garish, as there are only two shades of green, and I am not very good at pixel art, so it doesn't look very good. However, when put in context to a potential desktop screen:

After implementing the image into UE4 to test the scaling, I realised I had made it at the wrong resolution of 32x32, instead of 64x64. If I had made it at 64x64 then I would've been able to include more detail which would make it look more realistic on a desktop.

After researching the colour palette more, I found that there isn't a set palette for Windows 2000, but that it uses the Image Color Management (ICM) 2.0. This means that I don't need to necessarily stick to a certain palette, although having one would be very helpful.

For the window that will hold the main menu's buttons, I created an icon for the top left of it in the title bar. It turned out very well, as I used the same colours as the title bar of Windows 2000's windows.

Problems with World Space Widgets

Because Skyhouse is a VR game, a 3D, World Space widget is needed, as Screen Space widgets aren't rendered in the headset. However, upon creating this widget and displaying it in an Actor with a Widget component, it doesn't keep its perfect pixel style like it does in Screen Space.

The pixel perfect UI in Screen Space.

The blurry, anti aliased UI in World Space.

After researching this for a while, Harrison, Jay and I couldn't find an answer to fix it. We tried adjusting the anti aliasing settings, changing the mipmaps of the images, the filters and compression settings, however it isn't an image issue but a rendering issue as shown with the text, which is also blurry. As such, something with the World Space rendering is anti aliasing it but we couldn't find a way to turn it off or adjust it in any way.

Because of this, I've had to make some changes to the original idea. The computer the UI will be on will be closer to the player and could also have a CRT effect which will help hide the blurriness. However this could also be visually annoying when using the UI, so will have to be a weak effect if it is to be used at all.

While working, our friend Jack came over to see if he could help us as well. He suggested some similar things that we had already tried but also some new things such as whether the engine was downscaling it to save load time or the viewport resolution used a lower resolution to the engine default. After some more deliberation, we came to the conclusion to try and scale up the entire widget to instead hide the blurriness behind the larger text and to then resize the widget in the level.

A very large example of how having a large UMG UI and scaling it down in the level makes the blurriness harder to see. This is very similar to having a pixelating filter on an image, and adding cells to it to artificially increase its resolution.

Looking at the difference between this higher resolution UI and the lower one, the difference is night and day.

After Harrison modelled a CRT monitor for the main menu UI, I arranged the menu to fit the screen, and added a background befitting of the Windows 2000 theme. However, when we came to test it, the menu was no longer interactable. Since I had placed it inside the collision box of the monitor, the monitor stopped the trace from the player getting to the UI, making it impossible to interact with. To solve this problem, Harrison changed the collision of the monitor's model from "simple" to "complex", which uses the model's polygons as the collision shape instead of a rough box around the model like what "simple" does. This means the collision doesn't obstruct the UI, and it becomes interactable again.

Task Bar

I created the task bar in Photoshop complete with the icon of the program I made as well as a knock-off Windows logo using inverted colours. I created the left side of it by itself and have a separate, tileable version of the task bar that I can repeat for the rest of it, allowing it to work at any size (very useful for this project as the workaround for the 3D widget blurriness is making it twice the size and scaling it down).

Because the start button is part of the image, it won't be functional. This is something I would've wanted to be functional as some sort of Easter Egg, but I ran out of time unfortunately, so had to settle with a still image.

The button I made previously used an incorrect colour palette, and so didn't work well with the window background. I remade the button with the correct palette, and because I had created my own Button Slate Class in UE4, I only had to change the images once and they would update on all the Widgets I had placed.

The screen looks very bright, which makes details hard to see, so I disabled auto exposure and added a simple reticule so you can see where you're pointing for testing, and got this result:

It's not great, but it's slightly better than before and is good enough. Another thing I could do is decrease the brightness, however I don't know how to do that.

Bibliography

Hospital Moodboard:

Southend Hospital Building Photo - Your Southend. (Unknown). Unknown. Available: http://www.yoursouthend.co.uk/stroke-victim-survives-thanks-swift-response-southend-hospital-medics/. Last accessed 9th Feb 2022.

Aintree University Hospital Photo - Gamp, J. (2020). Coronavirus: Two staff members at Liverpool hospital test positive as UK cases soar. Available: https://uk.news.yahoo.com/coronavirus-liverpool-aintree-hospital-positive-uk-confirmed-cases-151041020.html?guccounter=1&guce_referrer=aHR0cHM6Ly93d3cuYmluZy5jb20v&guce_referrer_sig=AQAAACt0P0FoiTnI1K. Last accessed 9th Feb 2022.

First Signs in Corridor image - Unknown. (Unknown). Hospital Signage. Available: http://www.mksign.in/printport.php?head=SG9zcGl0YWwgc2lnbmFnZQ==#. Last accessed 9th Feb 2022.

Second Signs in Corridor image (shown twice) - mtreasure. (2015). Image of hospital corridor with hanging signs / directions to wards stock photo. Available: https://www.istockphoto.com/photo/image-of-hospital-corridor-with-hanging-signs-directions-to-wards-gm469543388-62406642. Last accessed 9th Feb 2022.

Southend University Hospital Sign Image - Unknown. (Unknown). Southend University Hospital. Available: https://www.cenergist.com/southend-university-hospital. Last accessed 9th Feb 2022.

Orange Corridor Signs Image - Suzuki, P. (2015). Maternity hospital. Available: https://www.pinterest.co.uk/philipsousuke_j/maternity-hospital/. Last accessed 9th Feb 2022.

Last Blue Corridor Signs Image - Thomsen, P. (Unknown). When NHS Staff ‘Sign Hacked’ Their Own Hospitals. Available: https://www.pinterest.co.uk/pin/433541901623250797/. Last accessed 9th Feb 2022.

Colour Palette example image - Jheald. (2010). List of software palettes. Available: https://en.wikipedia.org/wiki/List_of_software_palettes. Last accessed 13th Feb 2022.

Windows 2000 Recycle Bin image - Unknown. (2021). old windows icons. Available: https://oldwindowsicons.tumblr.com/post/655273722329071617/windows-2000-recycle-bin-full?is_related_post=1. Last accessed 20th Feb 2022.