Something every designer would love to hear is,"This can make your job easier!", or "This will give you an unfair advantage!". Now that Artificial Intelligence is literally being used everywhere, I would love to share some of the AI-powered Figma plugins that I've used and tested personally that can help streamline your design workflow.

1. Ando AI

(Download the Plugin)

Ando AI helps you generate millions of design ideas from prompts, shapes, and images. This can help designers fill up their mid/high-fidelity wireframes with relevant custom images and visual assets that can be used to make your wireframes more beautiful. (P.S. The post image was created using Ando AI!)

2. SPELLL

(Download the Plugin)

Nothing hurts more than presenting your beautiful wireframes, and you catch spelling mistakes smack bang in the middle of your presentation. Embarrasing. It makes me wonder why companies dont make propreitary spell check for their design tools. As chronic mis-speller, this plugin is paramount to my design process. This will help a designer check spelling and even grammar and vocabulary, and then fix them with a single click.

3. MagiCopy

(Download the Plugin)

Tired of thinking of a jaw-dropping punchline for you landing page? With MagiCopy you're able to generate copy for your landing page according to the industry and product you are designing for. If you dont like generated copy, you can always use it as a guide or inspiration to finetune your finals punchline.

4. Stark

(Download the Plugin)

Stemming from my previous article about WCAG Compliance. One of the easiest ways to not be compliant is the incorrect use of colour. Stark is the Figma version of https://webaim.org/resources/contrastchecker/ where you can see real-time of the contrast score for colours and different sizes of text.

As time goes by and AI advancements in Figma Plugin progress I will update this article in the future on the useful plugins that I will end up using.

If you found these useful, please do share with your fellow designers.

Well, well, well, what do we have here? Another trend report on the exciting world of Artificial Intelligence? Oh boy, I can hardly contain my excitement. I mean, let's face it, nothing interesting at all has happened in the world of AI recently, nope, nothing at all. Just kidding, of course. AI is a constantly evolving field, and there's always something new and exciting happening.

One area of AI that has been making waves recently is the field of language generation, and more specifically, the development of GPT-3 based models. GPT-3, or Generative Pre-trained Transformer 3, is a state-of-the-art language generation model developed by OpenAI. The model has been trained on an enormous dataset of text, and it is able to generate human-like text with remarkable coherence and fluency. The results are so impressive, in fact, that some people have even dubbed GPT-3 the "AI writing assistant."

ChatGPT is a variation of GPT-3. It's designed to generate more natural, conversational text. It can understand the context of what is being said, allowing it to respond in a more human-like way. This ability makes ChatGPT a powerful tool for tasks such as chatbots, conversational AI, and language translation. With ChatGPT, you can generate text that reads as if it were written by a human, in most of the cases even more coherent and natural than text written by a human.

In this article, we will take a closer look at ChatGPT and explore its potential use cases and implications for a wide range of industries. From automating customer service chatbots to helping write news articles, ChatGPT is set to have a big impact on the way we interact with language-based technology in the future. So, sit back, relax, and enjoy the ride as we delve into the world of ChatGPT.

This opening section, of course, was entirely written by ChatGPT, responding to the prompt:

Write an intro to chatgpt for a TechLab trend report article, it should be conversational, some sarcasm and humour is ok, for example "noting interesting at all has happened in the world of AI recently, nope nothing at all". Trend report articles generally are informational for academics and university staff in fields other than technology.

ChatGPT is just one of many tools that have emerged in recent months, but it has certainly been one of the most discussed. From its use to refine and obfuscate malicious code, improve viruses to avoid detection, and advice on how to create explosives to undertake acts of terrorism. Though Open AI, the company behind ChatGPT, has taken steps in recent weeks to curb the "exploits" that grant access to more pernicious responses.

Indeed, the capacity for fun with the system is near limitless. I have had it developing content for emails, communications, of course this article, but also more creative endeavours like:

I also decided to use it for my own benefit in salary negotiations and came up with the following :

One of the benefits of ChatGPTs model is that it remembers all the things that you ask it. I wasn't happy with the rap above because I don't work in sales, so I provided it some more context:

I'll include a comprehensive gallery of other examples below, I have used it to write code, come up with business ideas, rewrite segments of text, come up with copy and monologues for my Dungeons and Dragons campaign, to generate a walking tour around Tokyo which I then asked it to convert to a leaflet.js map of points of interest and many more. Suffice to say for lots of tasks, ChatGPT will revolutionise your life, but it is important that you acknowledge the big elephant in the room. ChatGPT doesn't actually know what the truth is, but it will confidently tell you that it knows.

A warning for language, but the academic paper "On the reception and detection of pseudo-profound bull$#!t" sums it up nicely:

“It is impossible for someone to lie unless he thinks he knows the truth. Producing bull$#!t requires no such conviction.” – Harry Frankfurt

ChatGPT doesn't know the answer, so it can't lie. It just sometimes spouts bull crap. Behind the scenes, of course, we know that ML models of this nature generally have a confidence or uncertainty measure, and in future iterations this may be available to the consumer, but as of now you don't know when you are being told the truth, or a rough approximation of it. ChatGPT is currently free, so is attracting a lot of attention as people test it and push its limits, which can only lead to a better product in the long run, but it isn't the only AI tool that is making waves right now.

Generative Art is an area that is currently receiving mixed reviews. The quality of the outputs is unarguably marvellous, but the ethics of the model training leave a lot to be questioned. At the very least, you sometime see a ghostly signature on the generated image, at worst it's a direct copy. Among the best of the current offering is Midjourney, able to generate in a multitude of styles and at acceptable print resolutions, and delivered through a Discord bot that allows for generation of images anywhere. It is not limited by style and can master some very complex themes and narrative elements.

For work in presentations or to communicate inspiration with a designer or artist, this tool is incredibly powerful and has formed a large part of how I describe locations for my Dungeons and Dragons game as well as discussions with my tattoo artist about what styles I'm interested in, but at this stage I think it's best to avoid commercial applications, despite the licence allowing it.

In the last 3 months we have tried over 100 different AI tools across a huge range of areas, from Text to Speech, to Text to Video, to Music, to Translation and more, and each is worthy of its own Trend report article... but you can find an ever-growing list at https://www.futuretools.io/, but it's by no means exhaustive.

While Language Generation and Image Generation seem to be the most present, the coming year will be both exciting and terrifying as the machines start to take over. As we look towards the future, it will be interesting to see how these AI-powered  tools continue to evolve and how they will be used in various industries. It's exciting to think about all the possibilities, but it's also important to remember to tread carefully and use these tools for the betterment of humanity.

When I started my degree in Design Computing in 2017, I never really learned or talked about software accessibility or inclusive design. I only focused on the aesthetic aspects of UX/UI design, the only thing in my mind that I thought could give me the most marks. Now working professionally for 2 years at TechLab, and constantly deploying applications to the broader audience, inclusive design has been in the forefront of my design process, specifically methods on bringing WCAG compliance to our applications. I would like to share some of the processes and tools that go towards making applications WCAG compliant.

What is WCAG?

WCAG (Web Content Accessibility Guidelines) are defined by W3C (World Wide Web Consortium), an international community that develops online standards. This set of guidelines are meant to enhance mobile and web functionality to be friendlier and give end-users extreme flexibility. Due to the dynamic nature of mobile and web technologies, WCAG guidelines are regularly updated to meet those new standards. WCAG 2.0 is used mostly for web accessibility – but the latest version, 2.1 provides support of accessibility on mobile devices.

Requirements for making a site WCAG-compliant

To ensure that your application meets the WCAG standard, it must follow the four POUR design principles:

• Perceivable: Information and user interface components (interactive links, text boxes, buttons, and so on) must be presented in a way that all users can understand. If the applications useful content is completely imperceptible to any of its users, it fails the perceivability test.
• Operable: User interface components and navigation must be operable.
• Understandable: Information and the operation of user interface must be understandable. If a user cannot grasp how a website works or what its information means, it fails the understandability test.
• Robust: Content must be robust enough that it can be interpreted reliably by a wide variety of user agents, for example, not only standard web browsers but also third-party assistive technologies such as screen readers.

How do I make applications WCAG-compliant?

WCAG fortunately breaks down the POUR design principles into several lower-level guidelines for specific topics and further dissects each guideline into a set of success criteria, which can act as a checklist for compliance.

The success criteria are also classified under three levels: A, AA, and AAA, with A providing the most basic level of accessibility and AAA the most comprehensive. In most cases, I only meet the demands of WCAG compliance to up to AA. Though it’s beyond the scope of this article to list the guidelines and criteria in full, I use these checklist guides to make sure I am following all WCAG requirements.

• “How to Meet WCAG” quick reference guide
• WebAim's WCAG 2 Checklist

Tools for testing accessibility compliance

Accessibility tools for checking color contrast

1. https://webaim.org/resources/contrastchecker/ is the most basic tool in terms of checking color contrast. I use this tool the most as it influences me on the use of color, early in the design process, to check on things such as buttons, banners, images, etc.

Accessibility tools for checking the code

1. WAVE Browser extension for Chrome and Firefox will check a range of features, including color contrast, the use of alt text, Accessible Rich Internet Applications (ARIA) labels, form features, Document Object Model (DOM) structure, missing titles, duplicate content and much more. The tool also provides tips on how to fix the found errors and references to accessibility guidelines too. We find it very useful for quickly checking heading hierarchy in the Outline view.
2. AChecker is an online tool for testing live websites, which also provides results grouped by Guidelines.
3. Pericles is an online screen reader tool. I use this to check the order in which text is read.

Accessibility tools for testing keyboard navigation

1. Mozilla Firefox is a great tool for checking if the website can be used with a keyboard. To enable keyboard navigation, locate and enable the “Always use the cursor keys to navigate within pages” setting in your Settings or Preferences panel.

Designing for Dignity

Designing for dignity is about ensuring people of all abilities can access an application in a way that is equitable for all. It is about designing a workflow with the expectation that people can  access the application like everyone else, regardless of mental or physical disability. However, it is sometimes impossible to entirely gauge a user’s lived experiences. In terms of a recent TechLab project, PersonalRecoveryPlan, there are several factors that can contribute to an at-risk user to trigger a mental crisis. This means the application will have to take measures in supporting that user if that adversity strikes. These measures include designing coherent buttons and workflows for users to access external and internal resources that may assist their crisis.

Why does it matter?

Dignified access for all users allows them to access and navigate an application/website on the same basis as their colleagues and other users. To the point where they can do it independently. Taking these comprehensive measures represent the crucial ethos to build a better and more inclusive web. Presently this is most prevalent as 15% of the world’s population has some sort of disability and 1 in 8 people currently suffer from mental or neurological disorder (WHO).

The University of Sydney also has a large presence in improving access and inclusion for people with disabilities. This is arranged in the Disability Inclusion Access Plan which is reviewed and renewed every 6 years. The Disability Inclusion Access Plan is a strategy for changing practices which may result in the discrimination against people with disability. It identifies and removes barriers for people with disability to fully participate at the University, including its learning, teaching, physical, digital, living and communication environments. Digitally, the plan saw the roll-out of accessible communications technology across the University, including the installation of accessibility software across student computing spaces, and all University of Sydney web pages being progressively updated to meet WCAG standards.

As an innovative team and environment at Techlab, we don’t only pave the way in emerging technologies and processes, but we also sustain the values that The University of Sydney maintains in the space of inclusivity, dignity and accessibility in all our products.

Virtual tours have been in use in marketing and real estate as interactive, informative approaches to immersing customers in a space, primarily in sales and marketing. However, broader use cases in education and training continue to come to the fore.

Providing site-specific learning experiences at scale, both synchronously and asynchronously, while measuring the engagement of learners, is the most difficult challenge the pandemic has imposed on educators. With the implementation of digital tours described here, we can deliver rich and pedagogically sound experiences, expanding the reach of tertiary learning beyond the cohort of students able to attend in person. While assuring a baseline equitable experience for all students, these experiences are accessible from a browser or compatible with VR headsets.

It starts with photos and videos - the technology hasn't shifted massively in 2020-22, so as a starting point for some equipment employed here, you will want to reference What we've learned from 7 years of 360 video. Since that article was published, the Insta-360 one R has formed a solid foundation for our mixed reality and 360 platforms.

Virtual Site Visits

EPIC lab is a Research laboratory using 3D tech to improve global quality of care around the world, located at Children's Hospital at Westmead. As part of the Applied Medicine Major, students previously attended this site to learn about prototyping, product design, and regulatory compliance for medical devices. Hospital site visits were highly limited with the strict procedures introduced because of the COV_SARS 2 Pandemic. https://epic.techlab.works was developed across a few days to allow students to visit the laboratory virtually and see some of the technology deployed there. Combined with online discussion and tutorial participation via zoom, the lack of access was supplemented.

Tutorial Lab inductions

Since travel was restricted in early 2020, many students for the semester (and the 5 semesters since) were unable to travel to Sydney campus; this meant that in classes with heavy tutorial focused curriculum, students would be undertaking these studies virtually.  While some experiments and tutorial activities for electrical engineering students could use online toolkits, the actual environments of the labs are a core part of the study. Our solution to this was to create a similar virtual tour for each of the labs at https://eielabs.techlab.works/  highlighting the equipment that students would be using online simulated versions of.   We also included the floor plan of the J03 electrical engineering building, for context, and to connect the students to campus, as well as induction to points of safety.

Urban Ecology field trips online

Walking around for many hours to investigate places where people congregate was simply not viable during the pandemic, In Urban ecology, both undergrad and postgrad units involved groups of 150-400 students per semester traversing 12-15 km, walking in groups across different zones on campus. This entailed note-taking and producing reports (on buildings, sustainability and green areas) from these site visits for tutorial activities and assessments. While the pandemic a blocker for large groups to gather on campus, there was already a pre-existing inequity issue for students with disabilities or special needs to participate with the same level of rigour as their peers.

Museums and object based learning

The Chau Chak Wing museum opened in 2020, resplendent with nearly half a million collection items. Many academics had begun the work of integrating object-based learning into curriculum; museum interactions were slated to be a key part of multi-faculty graduate experiences. The dedicated learning spaces developed to allow students to interact and observe ancient artefacts, ethnographic exhibits, science and historic photography and contemporary artworks.

70% of the items on display in the CCWM were previously locked away in storage for 20+ years and sizeable resource investment went into having them ready for learning and research - the challenge and opportunity was to now have the online environment compliment the built environment. In order to have our students engage with the museum, we co-developed a tour of some of the more interesting artefacts with students, and tie them to their studies. We worked to produce multiple versions for Arts, Science and Business, plus a communal version the museum could use in their communications and outreach. E.g., https://usydcampus.techlab.works/ includes prompts for students to engage and undertake assessment.

Extending into training

With high-fidelity 360 video comes the opportunity to implement training and compliance. Ranging from Virtual Food Science labs, Dental Radiography, Cadigal Green land learning, to 'Farm to Table' Supply Chain, we have created immersive experiences to train students and even industry partners and embed them in spaces that would otherwise be difficult or impossible. E.g., the University has only four Dental Radiography suites for use by several hundred students. However, using VR, we can grant them access to quizzes and tests for self reflection with direct access to the spaces virtually at the same time.

Connectedness to campus

FASS1000 - a foundational unit for ALL commencing students of the faculty of Arts and Social Science, was originally conceived with an instructor led walkthroughs of different parts of campus; covering aspects like space, place, history, activism, culture and community; to build a sense of connectedness to the University of Sydney. Not only did we virtualise this experience, but also equipped students with the ability to create and share their own journey-map - THe Unit of study won the 2021 Vice Chancellor's award for innovation and excellence in teaching.

Evolution of teaching platforms in 3D

All previous examples share common barriers - students/educators cannot create them without significant technical training (e.g., React and JavaScript based tours, complex 3D modelling tool sets), or significant costs in licenses, in the case of the Museum and Radiography examples. 3D Vista software allows rendering 360 experiences, but still requires a Sysadmin to deploy it via AWS or other similar infrastructure. Additionally, while we might password-protect web services, or implement a web access gateway for students/staff to use their university credentials for access, we can't measure granular engagement by users for the purpose of auditing, or for assessment.

To that end, we have been increasing the utility of our in-house VR asset management tool set: Eta-Ki.

Eta-Ki now supports 360 image/video annotation, creating collections, linking to associated Assets, allowing a "virtual tour" to occur entirely inside the application. This avails the benefit of granular navigation and interaction metrics linked to each user, plus the ability to protect intellectual property behind the university identity system. Eta-Ki also supports gigapixel imagery and 25 formats of 3D models, which otherwise pose cross-platform compatibility challenges. Now extended it to generate gigapixel imagery from Microscopy slides, it is built entirely 'serverless' and already in use in 4 units of study, 6 more planning to go live in soon.

While bespoke solutions and off-the-shelf tools like 3D Vista help in the short term, they are not sustainable as a delivery of comprehensive XR interactions, As Eta-Ki grows, we hope to have that solution in place enabling students, educators, Academics and community members to experience the University of Sydney, virtually.

Unity is a game development engine that is not only leveraged by AAA Game Studios, but increasingly by other industries as a communication and interaction platform. Thanks to the Unity Academic Alliance, we took the opportunity to integrate it into learning and teaching and trial it as a pedagogical tool, empowering the Medical Science curriculum through creativity and digital Fluency.

While digital literacy is important, the leaders of tomorrow will require digital fluency – the ability to navigate and utilise software for creativity, storytelling and problem-solving, and set the foundation for continued learning outside of their academic degrees.

For the past 2 years, we have been evolving a game development curriculum inside medical sciences. There are 3 reasons we have undertaken this journey;

1. Graduate outcomes:
   The University of Sydney's Education Strategy propounds a list of graduate qualities, among which are inventiveness, critical thinking and problem-solving, information and digital literacy and interdisciplinary effectiveness. We believe that game design and development greatly supports these capabilities.
2. The memory palace technique:
   Modern research shows that high performing mental athletes, the sort of individuals who engage in trivia competitions and game shows as a career, learn new information; they engage regions of the brain known to be involved in two specific tasks: visual memory and spatial navigation. (insert graphic of mind palace).
   They deliberately and purposefully encode the data with a visual representation and location within a kind of memory palace.
   We believe that visually structuring learning and making it interactive makes it 'sticky' in the minds of learners.
3. Learning should be interesting, immersive and challenging
   The COVID pandemic has reinforced the fact that the 120-minute lecture is not conducive to engaged learning. Research on lecture watching statistics worldwide continue to trend towards shorter more involved and impactful chunks of information, and participation and co-creation as the true drivers of knowledge retention. We believe game design and development to be one of the most participatory learning experiences a student can undertake.

The Challenge:
Medical science students are not game developers, most have never heard of Unity and their experience even with software development is limited.

We scaffolded the game capability across 3 years, in the first instance giving them heavily templated projects to work on to increase familiarity with the tools.

At first, we simply provided a template space for students to populate. This was loosely based on the shape of our existing Chau Chuk Wing museum, and would be recognisable to them. This was moderately effective, and introduced constraints and barriers. With a very diverse and talented cohort, the learning impact was varied, as it set a level of expectation but also made it difficult for students who wanted to flex their creativity. With that in mind, we began a capstone program with our Computer Science students to simplify the flow from web based architectural design, to Unity based interactive experience development.

Currently, we have limited exposure to Unity in the second year of the course, but we are working towards making it holistic. Throughout their second year, medical science students are exposed to a lot of complex information around anatomy, histology, pathology and a myriad of other challenging subjects. We have begun the work of creating 3D assets for a lot of these subject areas and storing them in our virtual asset management system Eta-Ki. While not limited to the medical sciences, our system caters to a massive number of file formats and allows for sharing and embedding in the University LMS system.

Third year is when the real challenging work begins: the capstone experience. This year, we really dialled in the approach for having students develop Interactive Unity experiences in small teams. The outcomes speak for themselves in terms of quality, but more so the student experience described in their own reflections and reports paints an engaged learning cohort, that understand the cross disciplinary application of the skills they have developed.

Furthermore, we have qualitative feedback indicating the value of this approach. Students entering the workforce with these skills contributing to their employability, and understanding that all jobs are technology jobs, from clinical roles, through to construction and facilities services.

These projects were delivered in a wide range of modalities, some projects were web based, some were games you installed, and still others were mobile applications designed to be used on the phone. As someone who has spent a lot of time making "apps", the speed at which real, high-quality products were delivered by non coders continues to impress me.

Moving forward we plan to expand this offering into other disciplines, as we understand that a driven and capable student cohort, will help create leaders in a range of verticals in the future, which will allow them to attack difficult and wicked problems from a truly cross disciplinary angle.

We also plan to continue to iterate the MedsUnited tool to better service the first year cohort. While the ability to code is important in the greater context of digital fluency, it's also important to give the students a sense of accomplishment, balancing this is where we will be focusing in the future. It feels very good to have created a real-world experience, just ask our students.

"We submitted our project a few days ago, and we're all so happy with what we were able to produce."

Just something I would like to also share with the teaching staff of MEDS1001, I had an interview for a full-time position at a medical company and one of the skill set they asked for was Unity. During the interview, I was able to show the interviewers our virtual gallery, and they were quite excited to see the virtual gallery and asked a few questions about the possibility of implementing VR. Tomorrow, I will be visiting their company to meet their engineers and attend their R&D meeting. Once again, thank you MEDS1001 teaching staff for preparing students for the graduate qualities and state of the art programming skills that the employers are looking for.

We are seeking permission to share these games with all of you, watch this space.

While TechLab team largely focuses on experimentation in emerging technology space, COVID created a sense of urgency to build digital and/or virtual experiences that replaced or supplemented in-person experiences in areas of learning, research and even student onboarding and support/allied services.

Not only did these need to be tactical (quick-fixes), but also adequately robust to be scalable with potential for continued use without significant investment of resources with technical know-how to support and maintain them.

In more simplistic, quick-builds, we started with applying our previously learned approaches in taking 360 images and videos and converting these to interactive virtual experiences. Key focus was also the AR, VR, MR, XR solutions to be accessible from any modern browser, to maintain equity across remote or onsite students.

In other cases, the builds ranged from complex 3D modelling of machinery for remote responsive use, multi-lingual platforms, Internet of Things (IoT) integrated tools for contactless health checks and more.

We don't have live links, but feel free to also ask us about these builds--

Playing games is fun - a great way to wind down, but more than that, there is a psychological drive in humans to play challenging games. We want to know that we have mastered a situation, and we enjoy the feeling of progressing and accomplishing goals.
This leads to the unique value proposition of educational games, or the genre of serious gaming, as a means to help the broader public come to grips with the world changing impact of Coronavirus. This approach has some proven success in Climate science, with high profile games such as Endling — communicating habitat destruction and extinction in a compelling and powerful way, with game play elements that are relatable to gamers.
The difference between climate change and the pandemic we find ourselves in, is that the former has 40 years of scientific consensus, often communicated effectively from scientists to laymen. SARS-CoV 2 is known to us only since about December 2019. Game developers do not yet have the same breadth of access and understanding, as the science of this pandemic has not been fully culturally assimilated yet.

So, let’s get medical scientists to design and build these games.

As part of the MEDS3888 Cohort in semester 2, 2020, we recruited 4 game development teams. The MEDS3888 program is designed around interdisciplinary problem-solving and our proposal was simple: How can you communicate the science of COVID-19 using video games?
In 2020, The University of Sydney Medical School joined the Unity Academic alliance; this gave us the ideal launchpad. Not only does this alliance give us access to software resources, but also  a huge community of global organisations to work with to deliver educational outcomes.

For the first 2 weeks of the semester, student teams spent time establishing a plan. We set groups up in Unity Teams and took them through our game development process.
Each group was taken through the 6-step process laid out by the “Ask a Game Dev” channel on YouTube. Then we (unit coordinators) setup Slack channels for each team, and brought in a design student intern 2 days a week to support debugging and polish for the teams. Through the process (which was 9 weeks) we asked teams to incorporate animation, sound design and world aesthetic every time they worked on their next component.

How do we get scientists to be game developers?

Each team spent 2-4 hours per week on more complex programming challenges with our design intern. Engagement on slack where possible was quick and responsive, however, we tried to rely on open source and credible online resources to solve the questions.  For anyone planning to rapidly build a game from never having done so before, that list is here;

1. Brackeys: Even though he has left Unity development on YouTube, His tutorials are still what any educator should aspire to.
2. Unity: Second only to Brackeys is of course Unity's own channel. The tutorials here are excellent and usually the most updated to keep up with the changes to the platform.
3. Thomas Brush: Renowned aesthetic designer in Unity for academia and industry
4. Dilmer Valecillow: Adept at the technical elements of Deployments and making cross-platform tools
5. Jason Weimann:  dedicated to online tutorials helping game development

Where content didn't exist, we created small video-based tutorials shared to every group, to be equitable. There weren’t many places where this was necessary as the Unity community is large and generous. Lastly, we encouraged groups to leverage pre-built assets as much as they could, as this was merely a 9-week development program, alongside full time study.

Enough already show me the games!

Gladly.
Three of these games are playable online, while one requires a download (please reach out to TechLab if you’d like to try ‘Mask On’).

Covoider by the Team Viral Gaming Dojo, is a swipe-based endless runner, where you social distance and collect PPE masks to protect yourself from the Virus. The message of the game is that a combination of approaches is necessary, as risk multiplies exponentially the more time you spend around other people.

UwUniverse: a message of hope by Team UwU_UwU is a First-person Shooter, where you sanitise the virus and deliver masks to the infected, to undertake simple tasks like collecting groceries or visiting the gym. It also addresses feelings of isolation for the elderly, as the coronavirus ravages aged care.

Infected by Team Crème de la Cookie is an immunology platformer where you travel inside the respiratory system of a SARS-CoV-2 sufferer to assist with their combatting the virus, by collecting the spike proteins necessary to develop antibodies.

Mask On is an endless runner game in an urban setting with a first-person persona, It has a free form and generative approach and requires sanitisation as a core approach to dealing with the virus. Handwashing and cleaning surfaces is the number one way this game addresses the concerns of COVID-19

The 3 web based games are available here. Remember these are student projects across 9 weeks, developed end-to-end, NOT by game development students, design students, or software engineers, but by scientists.
We believe that with 3 more weeks’ effort and some engineering know-how, each of these games could be released onto various app stores. To that end. We will likely be running more broad Med games challenges in 2021, and continuing this offering to the MEDS3888 cohort as well.

Banner Virus Image  created by Yuliya Stepkina as part of a MEDS3888 project using Cinema4D and protein structures from the Protein Data Bank.

The University is a unique operating environment to say the least, but there are elements we can learn from other enterprise organisations as well as those connected to educational institutions around the world, that set a gold standard for innovation and product evolution. These insights are drawn from 8 years of engagement with MIT Sloane, PWC, Gartner, Foresters and the NMC Horizon report, among many conversations with industry experts and Startups at every level of their organisational structure.

Analysis takes more work than experimentation :

It's important to emphasise learning from experiments and iterations on experiments over predictive analytics for innovation explorations, you can easily build 4 or 5 mock-ups or mini products with the time and money it would take to set a business analyst loose in the search for requirements. Even more challenging is the limited understanding of requirement analysis in companies that are not statistics shops. There are caveats to this, where you don't know the questions to ask, you can treat the entire predictive and prescriptive analysis approach as the experiment. This is what we did when we delivered the CPC-data framework.

The (sort of) myth of where good ideas come from:

While I am a firm believer in the coffee house hypothesis, good ideas are only half the story.

As enterprises like the University become more and more data driven, we must take a leaf from our academic colleagues and make hypothesis the centre of our experimentation. This isn't about proof of concepts, this is about minimum viable products to prove or disprove a hypothesis. For example, as we currently work towards an exhibition app for a current request, we've already disregarded 3 or 4 approaches via hypothesis. We are also implementing 2 or 3 others at this very moment. One of them (or more) will stick and be proven true, and we will implement it into the delivery pipeline for the project, making the outcome incrementally better for very little work.

The limits aren't the problem:

Learning the maximum amount possible with specific constraints – i.e., time, money, customer segment, technical implementation, etc.  –  is  the  goal and limits and constraints are there to be challenged. Often the reason for constraints in long-standing enterprise environments is lost to the mists of time, and healthy challenge of these constraints is part of the innovation pipeline.

From the Sloan report in 2016:

"This highlights a painful insight: The biggest challenges are not technical or financial, but cultural and organizational. At most firms, management overwhelmingly favors planning, programs, projects, and pilots over the real-world benefits of experimental knowledge and insight. Most don’t realize how exponential economics of experimentation can bolster their innovation investment portfolios."

"Executives frequently resist easy opportunities to cost-effectively experiment because they fear challenges to their hard-won professional intuitions and authority. Data-driven digital experiments might undermine pet hypotheses or business perspectives. But preserving, protecting, and defending the status quo may prove even more costly."

Make experiments social:

ICT don’t have  to own all experiments. Engaging diverse portfolios in the process, through Scrum, product pipeline engagement or even a chat or dialogues  around innovation, inviting comments and critiques, is a path to more holistic understanding of what needs to change. Broadly socialising results with vendors, community, students and more provides both much needed transparency and a feedback loop that adds value rather than criticism.

Prioritisation is hard:

I don't think I can overstate this. No matter what mechanism you use for prioritisation it must marry to organisational strategy. For the University there are 3 pillars, Research, Education and Culture, and the sub strategies under that give us broad brushes with which to paint our experiments. The actual challenge is not prioritising by strategic alignment it is more about balancing tensions that typically emerge between customer-facing managers and  technical managers about what to test first. Our Lab is unique in that we are both technical and customer facing, and we have been completely transparent about our experiment choices. You can see exactly the types of initiatives we engage in here, as can every stakeholder.

Perfection is the enemy of Done:

The Pareto Principle or the 80/20 Principle tells us we can't solve 100% of problems for 100% of users. In a product driven world, perfectionist engineering is suppressed in  favour of quicker, iterative sensibilities. You solve for 80% of people in the first instance. Outliers can be addressed with time and engagement, but if 80% of your challenge can be solved now, why aren't you doing it?

Understanding the Pareto Principle (The 80/20 Rule) – BetterExplained

BetterExplained

A focus on core outcomes as opposed to solving everything for everyone. If you manage expectations, people will understand. The University has a bias for "everyone's voice must be heard and acted on" this is in juxtaposition of the organisations capability and funding to deliver that level of outcome. Pivoting to "everyone's voice must be heard" is a better strategy for delivery of innovation, and even waterfall project engagement.

Measure trajectory not just outcomes:

Look, outcomes are important, but a shift in focus organisationally because of experiments with no outcome is almost as important. Helping to align the organisation to innovative and agile practices is as much an outcome as any tool or product delivered from an innovation portfolio.

Be Ethical:

'Don't Be Evil' and 'Do The Right Thing' are Alphabet Co and Googles Mottos and form part of their code of conduct. Set boundaries, and understand what data can mean. Engage with Cybersecurity specialists in design, and privacy experts in execution.

The Student Innovation Challenge is a fantastic opportunity for University of Sydney students to present their innovative solutions to real-world problems. There are over $28,000 worth of prizes that will be won!

Last year’s winners included a proactive management tool to combat UV risk in the workplace, rocketry research in the aerospace sector, and a digital tool that empowers underprivileged farmers use fertiliser effectively.

2020 categories and prizes

Start-up Innovation Prize - $10,000

Is your idea ready to launch? The Start-up Innovation Prize is for individual students, or teams of up to 5, who have developed a new idea with commercial potential. This competition is best suited for students who already have made demonstrable progress.

In addition to the cash prize, the winner will also receive a three month membership at the Sydney Knowledge Hub the University's coworking space for innovative businesses seeking to collaborate with the University. Membership includes access to:

• flexible workspace
• the Knowledge Hub community
• the programming and events held onsite
• facilitation to find and work with the brightest minds and other resources of the University to get your business to the next level.

Research Innovation Prize - $5,000

From theory to practice! This prize is for individual HDR students or teams of up to 5 who have discovered and developed a practical application for their research. This competition is best suited to students who demonstrate that their research has led to this outcome.

In addition to the cash prize, the winner will also receive a three month membership at the Sydney Knowledge Hub the University's coworking space for innovative businesses seeking to collaborate with the University. Membership includes access to:

• flexible workspace
• the Knowledge Hub community
• the programming and events held onsite
• facilitation to find and work with the brightest minds and other resources of the University to get your business to the next level.

Interdisciplinary Innovation Prize - $5,000

Two heads are better than one! Teams of 2-5 students working together from different disciplines can enter their solutions to real-world problems. The competition is best suited to students who illustrate how their disciplinary expertise have combined to produce a better idea.

People’s Choice Prize - $1,000 x 3

Shortlisted teams in each of these categories will have the chance to win the People’s Choice Prize. The idea with the most votes in their respective category will win $1,000. Make sure you spread the word to be in the best spot to win! Voting opens on the 27 July and closes on 12 August. Check back here on the 27 July for details.

How to Apply

Fill out the online application form.

The application for any innovation award should be uploaded as a 2-page pitch document in PDF format with an accompanying video or animated presentation (3 minute maximum). It should clearly explain the problem or opportunity on which the team has focused, and the solution that has been developed. It should present a strong case for the idea and how it fits the competition objective. Future targets and goals should be explained, as well as any traction the team has so far (e.g. prototype, funding, sales and partnerships).

Shortlisted teams will be announced on the 27 July 2020 and invited to present a virtual pitch to the Showcase Judging Panel on 12 August 2020. The winners for each category and the People’s Choice Prize will be announced on 26 August 2020.

Key dates

• Applications open: 18 May 2020
• Applications close: 25 June 2020
• Shortlisted teams announced: 27 July 2020
• Showcase Judging Panel: 12 August 2020
• Winners announced: 26 August 2020.

Judging Criteria

Data Privacy

Students should be aware that shortlisted applications will be publicly visible and should not include sensitive or confidential material. The application process, judging process and People’s Choice Award voting will be managed through the online platform Judgify. Judgify is not owned by the University. Students should review the Judgify privacy policy before submission. Students should keep copies of any material uploaded to Judgify for their own records.

Working from home, powered by technology - equipment to make your life easier.

Some basic equipment has been crucial to our business as usual approach.

Audio Equipment:

• Headset for audio only: Plantronics Black Wire 5220
• Wireless Microphone for audio capture: Samson USB Wireless Microphone
• Desktop Microphones: keep your eyes out for when these come back into stock, they are a game changer: Newer W-7000 USB microphone on arm

Video Equipment:

• Webcam for audio and video: Logitech C930e or Logitech Brio 4K
• Webcam for video: Logitech C930e or Logitech Brio 4K or Razer Kiyo
• On most computers you can adapt your Digital SLR to work as a webcam
• Tripod: Victiv 72-inch tripod or any tripod with a standard 1/4 inch screw mount will work, even a Joby Gorilla pod 3K can help with framing
• In a pinch EpocCam software can turn your Iphone or Android Phone, into a webcam

Document Broadcasting:

• If you need to share written notes, or show overhead capture, you may need a IPEVO VZ-R Document Camera which has a built in microphone.
• Second screens (even small ones) will make digital collaboration a lot easier.

Keyboard and Mouse:

• No specific recommendations, but a lighter weight mouse will help you avoid WHS related injuries.

Phones:

• Don't forget to redirect on campus phone lines, for some reason people still call them!
• The Service Now knowledge base has a lot of details on how to interact with your desk phone from home.

Files and Storage:

• Dropbox, Sharepoint and OneDrive are all good options, but if you find you and your team collaborate on a lot of documents, nothing beats DropBox Paper. The university has a full license.

AR/VR/XR initiatives in the time of self-isolation.

More Virtual reality headsets sold in the months of March and April than any time since consumer headsets were made available. This has been a combination of 2 key elements. The first obvious one is that with more people unable to leave the house, escapism to virtual worlds has become exceedingly appealing. The second is the release of Half-Life: Alyx. the first truly AAA game developed exclusively for VR. Estimates show that the game has sold almost a million copies, which exceeds most regular games, and is far and away the most downloaded VR game to date.In the Education side of things, we’ve been hard at work developing an Asset management tool for delivering 360 tours and 3-D model sharing for staff and students.  We call it Eta-Ki. The translation of “Eta-Ki?” from Bengali would be “what is this?”‌‌

‌‌‌This has helped us to consolidate an approach for media sharing between academics and students, and we hope will lead to more use in the future. It’s comforting that trends we have been tracking and prototyping with for the past seven years are now in very high demand for teaching and research. Headsets are still difficult, as we have only around 100 and they are all trapped on campus, but the price continues to drop and if Half Life: Alyx is any indication, there will be a lot in homes around the world for us to leveragewith our teaching content in the future.

The Tech Demo from Home.

There is a lot of equipment that we regularly demonstrate for educators and researchers, be it 3D printers, headsets, wearables, microcomputers or cameras. This has led to a lot of screencasts and video resource creation. It has also led to a lot of equipment in our homes! Each time one of us ducks to the lab, we seem to bring one more piece of equipment home so we can stream a demonstration for somebody, and It’s clogging up our work area – which we cannot recommend. Setting limits on what is possible is important here, but in theory, there is no limit to what can be demonstrated via a 360 video or even a High definition standard format recording.‌‌‌‌ a personal reflection from running the ANZAC day memorial live stream; it was eerie with four people at an event that usually has closer to a thousand attendees;  All of the tech demo was done from my balcony over the phone and over YouTube. You can even hear me chatting to my cat.‌‌

We Love Whiteboard.

Anyone who has a background in design will know the power of the whiteboard. There’s nothing so invigorating as prioritising requirements in a scrum on a large free-form canvas.  While there are plenty of digital white-boarding tools, mind-mapping tools and a myriad of multi-user editing tools available, this is a situation where we advocate the lo-fi solution.‌‌Write down the requirements (digitally or analog) and share it back to the group through a simple screen share. We’ve been using Dropbox paper in the first instance. But we’ve also been up-skilling our ‘customers’ in issue management.  Bringing them into the mindset of a milestone-driven developer makes their contribution participatory and leads to better results. Quite often now we wake to an issue in GitHub created by a user, that is articulated in a way our team can understand and action.

Virtually Available.

While Virtual Labs are nothing new, the level of Fidelity we can implement using Unity, WebGL and other modern delivery modalities means we can generate a lot of immersive and virtual content “Imagine what the University of Colorado do but on steroids.  A Disclaimer, I have a vested interest here developing pharmacological laboratories in VR in my outside of Sydney University role, with that said, I still see the value in the university adopting a broader approach. many skills can be on-boarded, audited and even certified via Virtual instruments.” Without getting too far into the pedagogy and without boring you with details of Blooms taxonomy, scaffolding, or productive failure, suffice to say the supporting literature for Virtual, Simulation and Immersion as powerful learning tools is growing every day. iDesign provides a great resource with practical examples but we can develop even higher fidelity solutions which, research says, increase the effectiveness of learning. Things like lab safety, complex and expensive instruments, and difficult locations are the obvious choices, but we have simulated mitochondria as big as buildings, and planets as small as marbles.‌‌‌‌

Up-skilling always.

Isolation is a great time to work on capability for the future. some members of ICT TechLab team have been engaged in the Unity Learning system, while we doubt the need to suggest to this audience that upskilling your team is important, we understand that austerity measures make budgets difficult. with that in mind, here are a list of free resources we have been engaging with to keep our team at the leading edge.

• The State government of NSW has made various short courses free
• Udacity are offering free nanodegrees. (A concept Universities may be quick to investigate following this pandemic)
• All University staff have access to LinkedIn learning which has a fairly broad number of short courses.
• General Assembly are running some of their boot-camps for free online
• Ribit is a digital platform that connects tertiary students to part-time paid employment, study related projects, and course accredited placements that focus on digital and STEM skills relevant to tertiary students’ study and future career aspirations. Ribit is Airtasker for student jobs.

Health and well-being – be inclusive and flexible.

Not going to imply we are all-rounders/experts in this area and still improving daily, but physical activity is something we all need, and mental health is more important than ever. There is a myriad of resources available to help manage your mental health while in isolation, and we recommend trial/using all of them. One thing we can say that has worked – is taking time to video chat with friends and colleagues after-hours about non-work things, and about work things. We used to vent Fridays at the pub or to our friends at dinner, now we need a way to do that.  For some of us venting is healthy. ‌‌Another unseen benefit of the startup culture we run in the TechLab is topping and tailing each day with a positive interaction among the team.  For most of the team, their first meeting of the day will be with their teammates to talk about the agenda for the day’s work, and the last meeting of the day will be to discuss anything that got missed, arising business and challenges to overcome. There’s something to be said for starting and ending the day with a group of people who are genuinely interested in each other’s wellbeing, and them making it known. All our team calls end with “stay safe” and it makes it feel like every day is RUOK day.

For Physical fitness in your lounge room, nothing beats this work out routine, recently published on reddit:

Conclusions and musing.

We may be through this in a few more weeks or months here in Australia. With the slow meander back to normalcy, or perhaps there is a relapse and we end up in this state for the rest of the year. It’s impossible to say. One thing however can be certain. when we do return to the Lab, we will be taking many of these digital practices with us.

When people talk or sing,  different muscles are being used, including some in the month and throat. Just like other muscles in human body, overuse of the ones that help human speak can lead to fatigue, strain and injury.

In Feb 2018. Google team published a paper, Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram, where they presented a neural text-to-speech model that learns to synthesise speech directly from (text, audio) pairs.

System setup

git clone https://github.sydney.edu.au/TechLab/tacotron.git

1. Build docker image

docker build -t nginx/tacotron2 .

In /tacotron

1. Run the built docker image (tacotron/tacotron2)

docker run --gpus all -it -p 8888:8888 nginx/tacotron2
cd tacotron2/

git submodule init; git submodule update --remote --merge

python waveglow/convert_model.py waveglow_256channels.pt waveglow_256channels_new.pt

jupyter notebook --ip 0.0.0.0 --no-browser --allow-root &

sed -i -- 's,DUMMY,LJSpeech-1.1/wavs,g' filelists/*.txt

Experiments

The solution from TechLab team is using the Tacotron 2 based on the Nvidia pytorch implementation of paper Natural TTS Synthesis By Conditioning Wavenet On Mel Spectrogram Predictions (J. Shen, et al.)

A deep dive on the audio with LibROSA

Install libraries

Firstly, let's install and import libraries such as librosa, matplotlib and numpy.

import librosa
import librosa.display
import matplotlib.pyplot as plt
import numpy as np

Loading in an audio file and plot the wave

# Load audio file
filename = 'output/chunk2.mp3'
y, sr = librosa.load(filename)

# Trim silent edges
speech, _ = librosa.effects.trim(y)

# Plot the wave
librosa.display.waveplot(speech, sr=sr)

Plot the Mel spectrogram

# Mel spectrogram
S = librosa.feature.melspectrogram(y=y, sr=sr, n_mels=128, fmax=8000)

plt.figure(figsize=(10, 4))
S_dB = librosa.power_to_db(S, ref=np.max)
librosa.display.specshow(S_dB, x_axis='time', y_axis='mel', sr=sr)  #default is fmax=sr/2
plt.colorbar(format='%+2.0f dB')
plt.title('Mel-frequency spectrogram')
plt.tight_layout()
plt.show()

Transfer Learning using a pre-trained model

Background

• Tacotron 2 is one of the most successful sequence-to-sequence models for text-to-speech, at the time of publication.
• The experiments delivered by TechLab
• Since we got a audio file of around 30 mins, the datasets we could derived from it was small. The appropriate  approach for this case is to start from the pre-trained Tacotron model (published by NVidia) which made use of the LJ Speech dataset for the training, and then fit to our small dataset.
  

In our experiments, there are several things to note:
-  Sampling rate differences: the transfer learning did not manage to work well if the sampling rate of custom audio is different from that of LJ Speech dataset. We should always convert the sampling rate of our own dataset to be identical with the sampling rate of dataset used in the pre-trained model.
- If we put a larger number of dropout, say 0.4-0.5, a surge of MEM occupation would happen and the mini-batch training would be stopped after several epochs. There are discussions on similar scenarios on GitHub and other platforms e.g. StackOverflow. People are prone to reckon it's due to the PyTouch architecture. The real cause of this scenario is still dim.
- A smaller batch size (e.g. 8 or 16) would lead to severe overfit.

Issues

However, the problem we were facing is the severe overfitting caused by the size of the dataset. Generally we can use some techniques to reduce the overfitting and do a better convergence:

• Enlarge the dataset by getting more data from Jason;
  
• Enlarge the dataset by using augmentation techniques on the audio sample provided (ref. to “Data Augmentation for Audio” below)
  
• Other machine learning/deep learning techniques, e.g. regularization, dropout, early stopping, and bigger batch size in per epoch (TechLab team has some limitation of using all the techniques listed here due to the GPU capabilities we have)
  

Future work

Data Augmentation for Audio

• Noise injection - add some random value into data (may help reduce overfitting)
• Shifting time - shift audio to left/right with a random second (our team had implemented this method to create more data samples)
• Changing pitch
• Changing speed
  

• The source code: https://github.sydney.edu.au/TechLab/tacotron

Reference:
- https://www.webmd.com/rheumatoid-arthritis/why-am-i-losing-my-voice

Since April 1, 2019 TechLab members have been reporting on how they spend their time in the lab. This report answers different questions, such as—which initiatives has the team been working on? Is it a meeting or a development work? Which faculty or unit are they engaged with? Over-all, this live reporting has now recorded:

Each initiative handled by TechLab is assessed with an impact score from 1 to 50. This allows the assessment of how the lab is allocating their time and maximising the benefits they could deliver with the limited resources.

The profile of hours reported by the Techlab team vary month per month. Multiple factors contribute to this including, but not limited to, semester activities, team headcount, and/or scheduled university shutdowns. With the varying schedule, initiatives are still prioritised appropriately; a comparison of monthly average impact and hours spent can be seen below.

As much as TechLab is composed of people with varying specialisation and expertise, its members also spend their time in varying types of activities and wearing different hats at work.

We live in a world where Corn and Sugar Cane has never been more important. I’m not talking about food consumption.

Looking back to my high school days in 2017, I took Chemistry for my HSC and we were introduced to Plastics. After endlessly studying and drawing a variety of plastics' chemical structures, I stumbled upon something called Polylactic Acid (PLA).

What makes PLA different from other plastics, is that it is derived from renewable resources. These renewable resources include sugar cane or corn starch. This makes PLA a “bioplastic” as it is created from biomass, also making it biodegradable! What this means is that PLA naturally degrades when exposed to the environment.

For example, a bottle made out of PLA that is left in the ocean would typically degrade in 6 to 24 months. Compared to conventional plastics, which would take several hundred to a thousand years to degrade.

Now why on Earth am I talking about PLA? Well, PLA is now used as filament material for 3D printing which is a form of Additive Manufacturing!

The benefits of PLA all apply to its use in 3D Printing. Additive manufacturing is essentially the process of building 3D objects by adding layer-upon-layer of material.

So now we know what PLA is and its use with Additive Manufacturing, what are its applications?

Education
At the University of Sydney, 3D printing has been widely adopted and used. There are makerspaces all over campus, that allow Students to come in and print a 3D model. Subjects are now teaching students how to use CAD software to design their own objects for assessment work and print them! An example is a few students from my degree, Bachelor of Design Computing; who participated in a Biodesign Challenge. They created a Smart Bandage applicator solution that was 3D printed as a prototype and showcased in New York!

Household
Additive manufacturing and 3D printing are now readily available for your everyday person! A 3D printer that is budget-appropriate and home ready like the Creality Ender 3 is at $320. You’ll be able to set up your 3D printer in no time and get started printing your favourite objects!

Health and Medicine
An example I want to show that is quite amazing is this:

It is a 3D printed arm attached to a motor that is controlled by an arduino with a muscle sensor. As the person flexes their muscle, the arm clenches and as the person releases, the arm also releases. I worked on this project with Jim Cook and it was 10 lines of code. Imagine the practicality and use of this on an industry-scale.

That’s it from me, hope you learnt something from this!

Our first 360 camera was 4 Elmo QBic MS-1’s in a plastic rig that positioned them for overlap. It was unwieldy, low resolution, required manual stitching in proprietary software, and at the time, difficult to import into a VR/XR Game development suite like Unity. We struggled to get good footage, but one group of students managed to make a pretty good lab tour, across a whole semester.

After the stitching nightmare of our first rig, we were looking for an all in one solution. This bought us to the 360 fly, much cheaper and in a much nicer form factor. We captured a lot of footage with this camera, though it only supported a 240 Degree Vertical field of view, careful production choices and positioning gave some impressive results, discounting this nausea inducing adventure (don’t have rapid janky movement in Virtual Reality).

The resolution though, as you can see, left a lot to be desired. We tried the Ricoh Theta 360, the Samsung Gear 360, and the Insta 360 Nano; all were fine on stitching but lacking in resolution.

Our next camera was the Nikon Key Mission 360. which came with 3 key benefits.

1. 4K resolution with onboard stitching.
2. Waterproof and submersible.
3. Sub $500 AUD price point.

For a long time this was our go to camera and still has applications due to its waterproof nature. The quality was excellent but there was still on camera stitching that wasn’t too easy to control.

You’ll notice in the clip above that  there are very obvious seam lines on the cabinet to your right and the chair to the left. These are exacerbated by the bright light coming through the stained glass in the Carillion room. A lesson learned, sometimes you don't get to chose your location, especially when developing education or documentary material, you will have limited control over the light.

This brings us to our current camera. Once we started to enact the University's VR strategy, we knew we needed to be able to control all of the variables. we came up with a list of requirements:

1. Maximum resolution, the bigger the better, with detail being the goal;
2. 3D capability (more on this later);
3. 3D Spacial audio;
4. Automatic Stitching that could also be tuned;
5. Long range remote control (you will notice I’m in the room with the Carillion above by necessity);
6. Full control of Video settings and real time preview;
7. Stabilisation;
8. Live streaming capability at high resolutions.

There were maybe 3 or 4 cameras on the market that met the requirements, however after extensive research we landed on the InstaPro 360 2. There’s nothing quite like 8K 3D VR at 60 FPS.

This camera has become our mainstay, from the work we do in food science with Lion Nathan and One Harvest, or filming break through medical facilities like the CPC hybrid theater,  MRI-Linac or Heart tissue regeneration lab at Westmead hospital. It has journeyed to events and the content from this camera is consumed across the curriculum. However, it is expensive and that makes it hard to scale. Right now, we train and onboard users in its use, and do much of the post production ourselves.

This is where the Insta 360 One R twin may come in - around 1/10th of the cost, with a tiny form factor. This might be the balance between professional quality 360 content within a University budget. The 3m selfie stick means this can be used to shoot in all sorts of circumstance, the weight is astounding and the built in AI for object tracking is truly astounding in such a small footprint. Seams are all but invisible, and the stabilization is wonderful.

There is no doubt we will see more and more of this in classrooms over the coming years, immersive learning is becoming the norm, and this technology allows us to provide high fidelity experiences in environment that may be unsuitable or unsafe for students. While ideally students can experience workplaces and labs directly, this is the next best thing to providing insight into other environments when we have cohorts of 200 or 2000. Thanks to the ease of use, and price of equipment, 360 Video delivered in VR or online is now a sustainable solution and our university among others will adopt a VR/XR strategy to leverage it.

But what does that mean is coming soon? Go-pro has the Max on the market, and its a worthwhile competitor, but where they both fall down long term is battery life.  with a 256GB card, you are looking at several hours of footage, with a spare battery you can maybe run the device for 2 hours, editing and previewing on your phone is tedious and also drains its battery.

Battery is the big blocker for a lot of the technologies we work with in the TechLab, wearables, XR Headsets, Mobile Phones, and Cameras will all be improved by the next generation of battery. Lithium Sulfur is promising but is still several years away, Lithium solid state will arrive sooner and this is being driven by safety concerns, thanks to airlines banning Li-Po and Li-ion in luggage among other similar situations. Without delving too far into the future, ambient electric harvesting of radio waves and human nano-generators are all coming down the pipeline though are likely 10 years away.

For now, it's a pleasure to use both the Insta 360 Pro 2 and the Insta One R Twin. Summing up the lessons of the past 7 years I’d say our real lessons are with scalability and process, while previously the TechLab or Immersive Learning Lab had to be greatly involved in order to ensure quality production, now with 5 minutes training anyone can produce a 360 educational video easily and cheaply.