Creating ERIN: The Rise of a Fully Interactive Virtual Receptionist

You exit the elevator and go through the open double doors that lead to the lobby ahead of you where you see a receptionist. When you enter, she lifts her head, looks you right in the eyes, and smiles warmly. As you approach her desk, she says "Good morning, nice to see you again! How can I help you today?" You tell her you're here to see Pete Slade this time, and after checking with him, she encourages you to take a seat while you wait. 

 

Walking through the lobby, looking at the awards and technology showcases on display, you notice she’s watching, following you with her eyes. She asks if you'd like something to drink, and a moment later a young man comes through a door with a bottle of water. He's the first human being you've met since you entered the building.

 

This is the story of how we created ERIN, a groundbreaking virtual receptionist, at a time when such innovations were still in their infancy. Our journey with ERIN, driven by the fusion of creative vision and advanced technology, marked a significant milestone for our team in the realm of artificial intelligence. In this piece, I'll take you through the challenges and triumphs of developing ERIN, illustrating how this endeavor not only shaped our understanding of AI but also laid the groundwork for future advancements in cybersecurity.

ERIN quickly evolved beyond a mere novelty. Taking charge of the front desk at Nitro, ERIN garnered attention for her ability to efficiently handle a diverse range of tasks. This project wasn’t just a testament to our team's skill and creativity; it also served as a tangible demonstration of AI's broader applications in everyday business operations. People began visiting the company specifically to interact with ERIN, and building management would often stop by with prospective new lessees to showcase her capabilities. Eventually, ERIN's growing reputation caught the media's attention, leading to visits and coverage.

Reflecting on the AI Revolution: From Concept to Reality

In the whirlwind of today's AI advancements, it's easy to get swept up in the constant stream of breakthroughs and futuristic applications. But sometimes, looking back is just as important as looking forward. 

 

As we stand on the precipice of a new era in artificial intelligence, I find myself reminiscing about the pioneering days of 2015 and 2016, almost a decade ago, when my team and I were deeply immersed in the development of a new technology, known as NitroServer®, which combined AI, Machine Learning, and featured innovations in data processing and connectivity. NitroServer embodied features and concepts that, surprisingly, still haven't fully trickled into the mainstream.

 

In this piece, I'll share insights from that pivotal time, detailing the work we did, the challenges we faced, and the innovations we brought to life. Our journey through these early days of AI reveals the conceptual groundwork and innovative thinking that later became instrumental in our shift towards using AI and machine learning for detecting behavioral anomalies in cybersecurity. The foundational ideas we developed while creating NitroServer and, subsequently, ERIN not only shaped our approach but also paved the way for significant advancements in the field.

 

First, let's start with a little background – Don't worry, this isn't a sales pitch for a product, nor is it crafted to induce a sale. This technology is no longer available. But, it is useful to explain what it did and the different pieces that comprised it.

NitroServer: Pioneering Server Technology for a Connected World

Back in 2015, the digital landscape was rapidly evolving. Amidst this transformation, my team and I created NitroServer, a server-based software technology designed to be at the forefront of this change. NitroServer wasn't just another piece of tech; it was a gateway to a new era of connectivity and intelligence for our customers. At a high level, NitroServer had four core areas:

  1. API Development and Connectivity: NitroServer was built with a clear vision – to facilitate the development of APIs and ensure seamless connectivity to diverse data sources. At a time when mobile applications were exploding onto the scene, NitroServer provided an essential platform for developers. It enabled them to rapidly create and deploy RESTful APIs, which were crucial for integrating different systems and making data easily accessible.
  2. Dynamic Clustering and Load Balancing: One of NitroServer's standout features was its ability to support dynamic clustering and load balancing. This capability ensured that as demand increased, NitroServer could efficiently distribute the load, maintaining high performance and reliability. This feature was critical in handling the surge in mobile app usage and the increasing need for scalable solutions.
  3. A Foundation for Future Integration: NitroServer's role extended beyond just facilitating API development and data connectivity. It served as a foundational technology, enabling customers to not only bring their technologies into the mobile age but also to integrate various systems in a seamless and efficient manner.
  4. AI/ML Capabilities: Recognizing the central role of NitroServer in our clients' technological ecosystems, we built it so that it would support multiple engines of capability. This architectural decision was pivotal. One such engine we integrated into our platform was AI/ML, which allowed NitroServer to not only process and manage data but also to learn from it and make intelligent decisions. This integration marked a significant step towards the intelligent systems that are now commonplace in today's digital world. (If you're curious to learn more about AI and Machine Learning, please visit the Learning Hub page on this website.)

As I reflect on NitroServer's contributions, it's clear that it was more than just a server technology. It was a forerunner of the interconnected, intelligent digital landscape we navigate today. The concepts and capabilities we explored laid the groundwork for future advanced behavioral learning and cybersecurity solutions.

Exploring the AI Landscape: From Rule Engines to Neural Networks

In the formative years, the landscape of AI was a vast, uncharted territory, brimming with potential. Our exploration into this realm was marked by experimentation with a wide range of techniques, some nascent and others more established.

 

Initially, our approach was varied. We dabbled in techniques that, while not AI in the strictest sense, laid the groundwork for more advanced applications. This included rule engines – systems based on predefined logic and rules. These engines were powerful for certain applications, but their limitations in handling complex, dynamic scenarios were apparent. Alongside these preliminary explorations, we also delved into traditional AI methodologies. Bayesian theory, a statistical approach that dealt with probability inference, was one such method. It provided a more nuanced understanding of AI's potential but also highlighted the need for more adaptive and robust solutions.

 

Our journey eventually led us to the realm of neural networks. This marked a significant shift in our approach. Unlike rule-based systems, neural networks offer the ability to learn and adapt from data, uncovering patterns and insights that were previously inaccessible. Supervised neural networks allow us to train models with labeled data, while unsupervised neural networks open doors to discovering hidden structures in unlabeled data. Unlike other methods, supervised and unsupervised neural networks are math-intensive for both inference and training. We used GPUs to assist us in this task.

 

As a side note, this was an interesting time, as GPUs were actually difficult to acquire at that point in time. NVIDIA, a leading manufacturer of these processors, was selling almost all their inventory to bitcoin miners. Back then, you could still mine Bitcoin using high-end GPUs, and there was a surge in demand as people were buying them up for mining rather than for AI/ML purposes. This created a unique challenge for us in procuring the necessary hardware for our different neural network projects.

 

Our transition to neural networks was not just a technological shift but a conceptual one. It moved us closer to the AI-driven future we envisioned – one where systems could not only process vast amounts of information but also learn and evolve from it.

The Genesis of ERIN

While all this work was going on, Nitro's business was expanding, and our existing office lease was coming to an end. We found and secured new office space in the heart of downtown that would be large enough for the additional employees we needed, and this new office space presented us with a unique opportunity. The new office, larger and more segmented than our previous space, featured a separate reception area. This space, though necessary, often felt isolated. It sparked an idea, playful at first, that evolved into a groundbreaking project. We asked ourselves: could we create a virtual presence to handle the tasks of a human receptionist?

 

The concept was simple, yet ambitious. Most visitors to our office shared predictable behaviors and patterns with visitors to offices universally. Some were repeat or returning visitors, while others were new, arriving for the first time. Their reasons were varied:

  • To deliver packages or food. (Some deliveries required signatures).
  • Employment interviews.
  • Regular visits from customers or partners.
  • Impromptu visits from the public.

Taking all this into account, we began brainstorming, envisioning what a virtual receptionist could do. A core pillar quickly emerged: any virtual assistant needed to offer human-like interaction.

 

Instead of a simpler approach, for example, a touch screen experience, we favored providing our visitors with the simulation of a human interaction. Our vision was to use the best of current technology and have it visually appear as if a person was sitting behind a desk, meeting the familiar expectation of a visitor walking into an office reception area. The virtual receptionist should follow socially accepted conventions of acknowledgment: make eye contact with visitors and adjust its line of vision so that it would look at a visitor wherever they stood in the reception area. In addition, our virtual assistant would need to graciously handle multiple visitors simultaneously by shifting its gaze as necessary between different people, identifying different speakers, and responding accordingly. Moreover, we envisioned our virtual receptionist to be capable of using natural voice communication, allowing for clear and mutual understanding between it and the visitors.

 

This was already ambitious, but we were just getting started. Additional requirements we captured included:

  • The virtual receptionist should be aware of people's positions in 3D space, recognizing movements left, right, toward, or away, and adapt their interactions accordingly.
  • The virtual receptionist should learn to recognize frequent visitors, similar to how a real receptionist would recognize familiar faces, such as package delivery drivers, customers, and partners.
  • The virtual receptionist should interact with company employees, notifying them when someone arrives for a meeting or visit.
  • The virtual receptionist would need to stay informed about general topics such as recent events in the news and the weather to allow it to engage in conversation with visitors. It would also be able to explain artifacts of old technology that were displayed in the reception area as a museum exhibit.
  • As the reception area was unmanned, the virtual receptionist would need to serve as the eyes and ears for the physical office after business hours, detecting any abnormal activities or visitors exhibiting suspicious behavior.
  • The virtual assistant would need to be aware of which employees were in the office on any given day, have access to their calendars, and use meeting schedules to gauge whether a visitor was expected or not.

We knew that whatever we developed would be driven by NitroServer, yet it would be unique enough to necessitate its own name. After some discussion, we chose 'ERIN' as an apt name for our created character, providing an easy-to-remember, human-relatable name for what would be the avatar interacting with visitors. This choice also cleverly doubled as a throwback acronym, referencing the underlying server and services.

 

ERIN, which stands for 'Enterprise Ready Intelligent NitroServer,' really did some heavy lifting with its acronym. It wasn't just about representing our ultimate platform goal, but it also carried a more lighthearted backstory. The name was a nod to Erin, a colleague who occasionally helped at the front desk, a role she wasn’t particularly keen on given all the other important duties she managed in her department. Therefore, she was more than happy to hand over the reins of the receptionist duties to a virtual counterpart. So, when our virtual ERIN successfully took over, allowing the real Erin to avoid the reception area, we knew our solution had hit the mark.

 

Even by today's standards, these objectives were ambitious. I am proud to report that we achieved them, delivering a capability that attracted media attention and became an excellent showcase, conversation piece, and technical triumph for everyone involved. Here’s the journey we took, along with some lessons learned, and more.

Technical Architecture of ERIN

ERIN represented the integration of numerous advanced technologies, each playing a vital role in her functionality. Balancing performance and minimizing latency across multiple machines was a key aspect of her design. For instance, one machine was dedicated to converting voice to text, which then relayed information to the NitroServer machine. Another machine managed her visual display, handling requests and initiating animations. Additionally, there was a machine specifically for processing vision and facial recognition.

 

This intricate setup was further supported by an array of sensors, microphones, and cameras, each contributing to ERIN's ability to interact and respond accurately to her environment. The image below is annotated to illustrate some of these key components, providing a visual representation of the complex system that brought ERIN to life.

 

At the time, various cloud-based voice-to-text translators were available, but they typically processed audio in batches and then returned the text. This approach, while functional, introduced undesirable latency, both in processing time and in capturing the visitor's spoken words in real-time. To avoid these delays and ensure we didn't miss any immediate context from the visitor's speech, we opted to create a streaming pipeline that ran locally and performed the voice-to-text process. This delivered real-time streaming translation, enabling the system to start processing the visitor's intent as swiftly as possible.

As the reception area was ERIN's domain, we strategically positioned cameras and sensors to enable her to observe all visitors to the office. There was no other way to enter except by passing her. Upon exiting the elevator on our floor and turning left, visitors would immediately notice ERIN, situated just beyond the open doors of our office, appearing to be seated behind the desk. The image below, shows the specific areas we wanted ERIN to monitor and respond to.

In the picture above, indicates the placement of cameras and motion sensors (both depth and infrared), positioned above and slightly behind ERIN's avatar. These devices were designed to track faces, learn facial features, observe body movements, and measure depth/distance.

 

In quiet moments, ERIN would seem busy with her tasks, her eyes moving as if focusing on something specific. However, when a visitor entered zone , she would look up, smile, and start face tracking to maintain focus on the individual. In cases where multiple people were present, she could track several faces simultaneously, skillfully shifting her gaze among them. As a visitor progressed into zone , moving forward, ERIN would offer a dynamically generated greeting such as, “Good morning, can I help you?” Should she recognize the visitor, the greeting would be more personalized: “Good morning, [Visitor's Name], nice to see you again. How can I help you today?"

Crafting ERIN's Avatar: Balancing Humanity and Technology

Designing an avatar for ERIN presented a unique set of challenges, particularly in the realm of simulating human features. The goal was to create a character that was approachable and relatable, yet without crossing into the uncanny valley – a term used to describe humanoid objects that eerily resemble humans but aren't quite lifelike, often causing unease or mistrust among real humans. Achieving this balance was crucial. We aimed for ERIN to be recognizably non-human, yet sufficiently human-like to facilitate natural interactions.

 

To accomplish this, ERIN was crafted in Unity, a powerful game development platform known for its versatility in creating complex animations and interactive experiences. We treated ERIN as a modular entity, with a distinct component designed specifically for her visual representation. This component communicated with a socket server capable of receiving detailed animation commands. These commands included both fully sequenced actions for complex movements and simpler start-stop animations for more subtle expressions.

 

One of the intricate aspects of ERIN's design was synchronizing lip movements with speech. This was handled by dividing the tasks between different components. While one module was responsible for the actual speech delivery, another was in charge of the corresponding visual animation, such as moving lips. We utilized UDP (User Datagram Protocol) packets for this communication, ensuring real-time responsiveness. A packet would be sent to initiate the lip movement animation as ERIN began to speak, and another packet would signal the end of the speech, stopping the animation accordingly. This level of detail was necessary to create a virtual receptionist that could interact with visitors in a realistic and engaging manner, making the experience as natural as possible.

 

As one might expect, ERIN underwent several iterations as we experimented with various designs and features. In the early stages, we used a portrait-oriented LCD monitor for her display. However, in pursuit of a more lifelike appearance, later versions incorporated a cut-out board combined with a short-throw projector. The brief video clip below showcases these improvements, including ERIN's head tracking feature, as she follows my movements from left to right. Although not visible in this clip, if I waved at her, she would respond by waving back. Additionally, her design included subtle, human-like behaviors such as natural blinking and occasional smiling, enhancing her realism and approachability.

Implementing a short-throw projector for ERIN marked a significant upgrade, enhancing her presence in the office space. This technology allowed her to appear more naturally integrated into her surroundings, providing a more realistic and engaging visual experience where visitors could see around her.

Face Tracking

Face tracking was a pivotal aspect of ERIN's interactive experience. She was equipped with facial recognition capabilities, allowing her to begin learning and memorizing the unique characteristics of each visitor. 

 

We employed various models to accurately track different face profiles, ensuring a high level of precision. ERIN was adept at recognizing when packages were being carried. She could correlate a familiar face, known from past deliveries, with repeat visitors. When a delivery arrived, ERIN would promptly notify the relevant person that a package was there. Additionally, she could interact with the delivery person, inquiring if a signature was required. Depending on the response, ERIN would then appropriately request an employee, who was on assist duties, to physically sign for the delivery.

 

The clip below demonstrates the technology in action, featuring one of our engineers, Alex. Note the green rectangle which frames Alex’s face as he moves. This indicates face detection and tracking in process. 

At the bottom of the clip, you can observe the system identifying and tracking specific facial features. This was crucial not only for facial recognition purposes, allowing ERIN to remember and recognize return visitors, but also for providing vital visual feedback to the interaction engines. For instance, ERIN could discern if a visitor was smiling, adding an extra layer of engagement and responsiveness to her interactions. 

 

The above clip also offers a broader view of our office's reception area as seen from ERIN's perspective. You can see the hallway extending ahead of her, as well as the elevator banks on either side. This layout not only situates ERIN as a central figure in the office but also demonstrates how her field of vision encompasses key areas of our workspace. 

 

Note the shelving on both the left and right sides of ERIN's view. This shelving housed a collection of technology museum pieces, some of which are quite valuable. ERIN served an important role in monitoring these items, effectively functioning as a vigilant guardian. She would alert us to any unusual activity outside of regular office hours when the office was typically empty. She understood when the cleaning crew was in the office after hours and their normal behavior, and was able to differentiate between “usual” and “unusual” occurrences.

 

ERIN could tell when the office was empty, but she could also determine who was present. Both things were made possible because of her integration with the office’s security and communication systems. ERIN had access to the card reader logs for the doors, enabling her to track who was entering or leaving the office. She was also connected to the Wi-Fi network, and this allowed her to determine who was present based on their device's connectivity. Additionally, she could refer to meeting calendars, further enhancing her ability to understand the office occupancy at any given time. This integration of various data sources allowed ERIN to learn the patterns of life of the employees within the office, and maintain a comprehensive awareness of the office environment, ensuring the security of the technology displays, especially after hours.

 

This aspect of the project involved the use of unsupervised machine learning – one of several areas where we applied these techniques. Training a machine to recognize specific people like our employees is straightforward when you know who they are in advance. If we had just wanted to identify our employees, we could have just trained a facial recognition system on their photographs and called it a day. However, the real challenge came from our desire to identify unknown new and repeat visitors. This required the system to collect in real-time the feature vectors of faces and perform unsupervised clustering on them to correlate similar faces into identities. In this case, as the system did not have photographs to guide facial recognition, it had to perform unsupervised clustering on them to correlate faces with identities. 

 

As we worked on this particular problem, we encountered the limitations of traditional facial recognition techniques that used hand-crafted feature-detection classifiers, such as Haar Cascade. It was at this juncture that we turned our attention to deep learning, inspired by Facebook’s DeepFace project. We discovered that deep neural networks allowed our system to produce feature vectors that were much more effective for the task of establishing identities.  We also learned that there were numerous challenges in building and maintaining clusters of them over time, which led us to explore the idea of treating these feature vectors as a time series, considering not just their individual characteristics but also how they evolved and related to each other over time.

Body Detection

In addition to face recognition, ERIN's capabilities extended to depth and distance tracking to monitor people entering or leaving the office. She was also equipped with sophisticated limb movement detection. This meant if someone waved at ERIN, she would recognize the gesture and wave back. Another engaging feature was her response to being photographed: if she detected someone pointing a camera at her, ERIN would smile, say “cheese” and pose with her hand up in a static wave, ensuring the photo was clear without any motion blur.

 

The image below illustrates these functionalities. It shows the system actively detecting movement and tracking the position of body limbs. In the initial phase, all limbs are clearly visible. However, as the person gets closer and the desk begins to block the lower body, the legs are highlighted in red. This shift in color indicates the system's switch from direct observation to predictive tracking, where it estimates the position of the obscured limbs.

This additional view further clarifies how the desk obscures the lower part of the visitor's body from ERIN's view. While we could have employed more sensors or altered their placement to continue tracking the feet, we determined that this level of detail wasn't necessary for ERIN's functionality. The existing setup provided her with all the necessary information to effectively track and observe a visitor, ensuring she could interact appropriately without the need for additional data on lower body movements.

Human Detection

The development of ERIN required a combination of multiple systems to achieve effective identification of visitors. While facial recognition was a key component, it wasn't solely relied upon, as visitors could potentially obscure their faces to avoid detection.

 

To build a more complete visitor profile, we incorporated electronic signature detection to make identification more robust. The ubiquitous presence of mobile devices, with their unique digital emissions, provided a wealth of information specific to individual visitors. By combining this electronic signature with the visual data from facial recognition and contextual information from meeting calendars, ERIN could create a much richer and more accurate profile of each visitor. This multifunctional approach was not just about preventing evasion; it was about crafting a more detailed understanding of each visitor, enabling ERIN to interact more effectively and personalize her responses.

Voice (Speaking and Listening)

Natural voice communication was essential for ERIN to engage effectively with visitors. This required the integration of advanced speech recognition and natural language processing technologies. These systems ensured that ERIN could not only comprehend spoken words but also respond in a manner that was both natural and human-like.

 

Our journey to perfect voice capture involved extensive experimentation with various microphones. The goal was to consistently capture streaming audio for real-time speech processing. Ultimately, we opted for high-quality, noise-reducing microphones, which we installed in the ceiling above the spot where a visitor would typically stand. These microphones were highly sensitive yet capable of filtering out background noise, such as the hum of air conditioning vents or air currents. The captured voice data was then converted to text and streamed into a natural language processing engine for interpretation.

 

For ERIN's speech output, we strategically placed speakers behind and slightly to each side of her avatar display. This placement ensured that the sound seemed to emanate from her 'head.' We designed the audio output to adapt between the left and right speakers based on where ERIN was 'looking' at any given moment. This setup created a dynamic audio experience, simulating the directional and projected quality of sound one would expect from a real person speaking.

Advanced Communication Capabilities

ERIN's role extended beyond mere visitor interaction; she acted as a vital communication bridge between guests and our team. To perform this role effectively, we integrated an internal messaging system within ERIN, enabling her to contact team members as needed. This integration was a cornerstone in establishing her efficiency and reliability as a virtual receptionist.

 

ERIN was given the names of employees. If a visitor mentioned the name of a particular employee, ERIN could then identify the member of staff whom the visitor wanted to see. She was also integrated with the office’s calendaring system, enabling her to anticipate who might be arriving for meetings. These integrations, all part of the NitroServer system, equipped ERIN with an impressive awareness of office activities.

 

For instance, if a visitor entered claiming to have an appointment with an employee, ERIN could verify this by checking the schedule. Upon asking for the visitor's name, she could cross-reference it with known appointments.

 

ERIN's method of informing employees about visitor arrivals showcased her ability to tailor her communications based on particular visitor situations. When a visitor arrived asking to see an employee without a scheduled appointment, ERIN would first inquire about the visitor's name. She would then send a Slack message to the employee, typically phrased as “A visitor is in reception asking to see you. Their name is [Visitor's Name].” This approach was distinct from situations where a visitor arrived for a prearranged meeting. In such cases, ERIN's message would align more closely with the scheduled context, for example, “Your 2 pm appointment with [Visitor's Name] has arrived. I have asked them to have a seat and they are waiting for you.”

 

The employees had the convenience of replying to ERIN with predefined options. These could range from “Option 1 - I am coming now” to “Option 2 - I will be there in a few minutes.” Based on their response, ERIN would communicate with the visitor accordingly, such as advising them to take a seat while they waited.

 

Furthermore, ERIN was programmed with thoughtful touches to enhance the visitor experience. For instance, she would inquire whether a visitor would like a bottle of water while waiting. A positive response would trigger ERIN to send a separate Slack message to the employee designated for assistance duties, asking them to bring a bottle of water to the reception area. This attention to detail in ERIN's communication protocol went beyond merely streamlining office interactions; it introduced a level of personal care and attentiveness into the visitor's experience.

Understanding Requests

Once the audio speech captured by the microphone was converted into text, the NitroServer AI engine took over to manage the context of the conversation with the recognized visitor. This capability was crucial, especially when multiple visitors arrived simultaneously, as ERIN needed to understand each visitor's specific purpose. Her interactions included the profile ID of each person, enabling the backend system to effectively switch contexts as needed.

 

A key feature of NitroServer’s AI components was the concept of 'personas'. This concept enabled nuanced interactions, allowing users to engage with the AI through a specialized chat window, similar to interacting with a chatbot like the modern-day ChatGPT. The use of personas was essential for preventing data leaks and restricting the scope of knowledge to relevant areas. For instance, a healthcare persona would be trained and knowledgeable only in healthcare-related topics.

 

ERIN was assigned a 'Receptionist' persona. We specifically trained her in this role, limiting her scope to interactions typical of an office environment and granting her access to data sources pertinent to her functions. In ERIN’s case, these included technologies like Slack, calendar schedules, and a selection of external internet data sources. These sources ranged from local and national news to local weather and included the capability to query internet resources, such as Wolfram Alpha, for certain types of questions she might have received. This specialized training and access allowed ERIN to function effectively and knowledgeably within her designated role as a virtual receptionist.

Tuning

To fine-tune ERIN's interactions within her environment, we developed a specialized mobile app. This tool was essential for setting the limits of her head movement and ensuring these movements aligned with the positions of people in the space. With the app, we could stand in the actual environment and accurately calibrate her responses in terms of direction (left, right) and depth.

 

The app also facilitated crucial tests such as animation checks and audio tuning. These functions were key to making sure ERIN's head movements looked natural when she appeared to be looking at someone and that her voice interactions were clear and precise. By using this app, we were able to coordinate and refine all aspects of ERIN’s functionalities, ensuring her seamless operation in the workspace.

Guidance and Privacy Measures

Some visitors were initially taken aback by ERIN, unsure of how to interact with her. Many couldn't believe that they could just speak normally, or they suspected some form of trickery. To alleviate this confusion, we added an explanatory helper sheet to the desk. This sheet briefly described what ERIN was and encouraged visitors to talk to her as they would to a human. Additionally, we incorporated a 'call for help' button, linked to one of the machines. When pressed, this button prompted ERIN to inform the visitor that someone would arrive shortly to assist them in person, and she would simultaneously send a request to the staff member on assistance duty.

 

The helper sheet also contained important legal language. It explained the terms of consent for visitors, covering aspects like the use of their likeness for training purposes, voice processing, and data storage. It also outlined the procedure for visitors to request the removal of their data, though this option was never exercised by any visitors.

Fun Experiences

The journey of creating ERIN was not just a technological endeavor but also a path filled with continuous learning and memorable experiences. Here, I’d like to share a couple of those amusing and notable moments.

Story 1 - A Prank

One such incident involved an engineer who, although not directly working on ERIN, was a good sport and a friend to the team. The office layout placed the restrooms in the hallway outside the office. There were two ways to access them: either through a private employee-only side door with a keycard system or by walking through the reception area where ERIN was stationed.

 

We decided to inject a bit of humor into the workday with this engineer, using ERIN’s capabilities for a playful prank. Since ERIN could detect the presence of people in her vicinity, we programmed a special animation just for him. We set up ERIN to recognize when Brian, the engineer, walked by her, ensuring she would only activate this animation if no one else was around. As Brian passed, ERIN would track him with her face and playfully 'flip him off.' 

 

This light-hearted prank brought much laughter to our team and remained a hidden, humorous feature in ERIN’s system for quite some time.

Story 2 - Wardrobe Malfunction

Another memorable incident was an accidental optical illusion relating to clothing – or the apparent lack thereof. The following images help illustrate this situation. In the first picture, you can see ERIN, with a black rectangle indicating the position of the desk in front of her. This desk, made of black marble, was highly reflective. Depending on the lighting conditions and viewing angle, it could create misleading visuals.

 

For a while, we noticed that some people would pause near the elevator entrance, staring towards the reception area but hesitating to approach. We initially thought they were simply intrigued or cautious about the technology and wondered why they didn't interact with ERIN. The mystery was only solved when a colleague's family visited the office. One of the children approached Alex, a lead engineer on the ERIN project, and innocently asked why ERIN wasn’t wearing a top.

 

We were baffled by the question because, as seen in the image below, ERIN was depicted like this:

However, once we positioned ourselves to mimic the angle and viewpoint of the onlookers, the cause of their hesitation became clear. The interplay of light and the glossy surface of the reception desk created an unexpected reflective illusion. The top part of ERIN's shirt, when reflected on the desk's surface, gave the illusion that she was dressed very inappropriately for the office, as seen in the image below. This unintended effect, resulting from the desk's high reflectivity and specific lighting conditions, explained why some visitors had a very different view of ERIN.

 

This amusing discovery prompted us to promptly update ERIN's wardrobe. We selected new clothing options that were 'reflection-proof,' ensuring they would maintain their intended appearance under any lighting conditions and angles.

Final Thoughts

ERIN's story is more than just a chapter in the history of AI for my team and me—it's a dynamic showcase of innovation. ERIN was not merely a technological breakthrough in and of itself; it represented a significant milestone in our quest to discover more and more powerful applications of AI and Machine Learning.  ERIN’s development and the lessons we learned were a major step forward toward achieving our present-day, pioneering advancements in cybersecurity.
 

Reflecting on ERIN's creation isn't solely about revisiting the past; it's about recognizing the impact of these foundational projects in shaping the future of AI. ERIN epitomizes the creative power that results when imagination is fused with reality and theory with application. The result is energy that propels us through the rapidly evolving landscape of AI.  She is both a testament to our team's creativity, dedication, and foresight, and a truly unforgettable part of our AI odyssey. 

 

NitroServer, years in the making, provided the robust platform upon which ERIN was built and symbolized a pinnacle of our collective efforts. Each team member's dedication, ingenuity, and unique passion formed an integral thread; woven together, they shaped NitroServer's and ERIN's success

 

Amidst this remarkable ensemble of people, I extend a special acknowledgment to Alex Matheu. As the lead engineer on this project, Alex was a cornerstone of our efforts, working alongside me at all hours. Together, we collectively designed and wrote code, assembled hardware, soldered circuits, ran cables and experimented with new ideas. His expertise and partnership were instrumental in navigating us through this endeavor and helping transform our ambitious vision into the success it became.

Pete
Pete Slade
January 6, 2024