EP. 50 Building Secure AI with Anthony Baio

Matt Pacheco
And welcome to the Cloud Currents podcast where we explore the strategies in tech shaping the future of cloud computing and cybersecurity. I'm your host Matt Pechio and I help businesses understand cloud trends to make smarter decisions about their IT strategy. Today's guest has one of the most hands on ground up journeys in tech you'll hear about on the show. Anthony Baio is the VP of AI and Machine Learning and Data Infrastructure at West Hills where he oversees a serverless GPT platform environment. And his background spans from offensive defensive security network operations, cloud engineering across multiple public clouds and a deep focus on generative AI and the risks and how to build systems responsibly around that. In today's episode we'll talk about his path into tech which is really interesting story.

A little bit about serverless cloud operations, security threats that come with AI adoption specifically. We know that's a topic that's on the top of mind for everyone listening and everyone everywhere right now in tech. And then also his thoughts on some of the intersection of cloud and AI and where that's all headed. So Anthony, welcome to Cloud Currents. Thanks for being on.

Anthony Baio
Matt, thanks for having me. I'm very excited to be on. Thank you so much for the intro.

Matt Pacheco
Yeah. So let's jump right in. Let's understand a little bit about you and your journey. Tell us where you started and how you got there into tech and cloud and what you're doing today.

01:54 – Anthony's Non-Traditional Journey into Tech

Anthony Baio
Yeah, absolutely. So I took a little bit of a non traditional path. So I, I started at the University of North Carolina at Charlotte. I was actually more happy to take a scholarship there to run track and field across country, then a little bit more education. At that point I started off as a double major in physics, particularly interested in theoretical physics as well as mathematics and then kind of quickly wearing on that physics is really just applied math. So not doing the quick double major with being a D1 athlete. Pretty complex schedule there. So really going heavy into the mathematics courses as well as doing minors in electrical engineering and computer science.

After I had completed all my mathematics courses towards my degree as well as electrical engineering and comp side, I realized I had about a year and a half worth of just Gen Ed courses which I'm more of a fast paced person and for me I didn't really want to entertain spending another year and a half in college just taking Gen Ed. I really wanted to get out there on the forefront and start applying knowledge to what I had known. So I actually went and pivoted and Joined the Air National Guard and kind of at the same time started pursuing more of the certifications in the tech space. So at that time Certified Ethical Hacker was relatively new, as well as the regular CompTIA certification such as the network plus security plus.

Those kind of certifications, which I've now learned, a lot of them actually bear more weight than a degree in this space. So after that, pursuing courses starting at the bottom, I think the bottom of the bottom in Staples, Easy Tech center, individuals just bringing in their laptops for a variety of reasons, downloading malware from Facebook or any kind of social media sites, things like that. So getting accustomed to more of the real world all, you know, while, you know, going home, just kind of locking myself in a room and going on YouTube, looking at Eli the computer guy and all these kind of free resources at the time, so widely available today and really getting myself ingrained in general concepts around networking and on prem environments and hardware. I was very into hardware.

So kind of buying as relatively poor at the time, cheap kind of hardware, what I could find on ebay, a lot of three and four year models behind a different type of firewalls. Palo Alto, many different brands out there, just picking them apart hardware wise, just trying to reverse engineer, see how each one of them work and then going to the network level, seeing functionality, things like that. So getting very much into the red team exercise of it on the, you know, how can I break in all the way, you know, down from the kind of layer one and then up to the application layer. I think a lot of things were different when I started breaking in, you know, 15, 14 years ago, where you had a lot more protocols going over unencrypted methods both at rest and in transit.

So maybe a little bit easier back then to do a man in the middle attack and things like that than it would be today. But was also interested in the defensive side as it seemed to foster a little bit more creativity and let's say from the mathematical side, just learning how attackers usually have one step up. So seeing how to kind of navigate that landscape to where from a security operation side, things that you can do to at least be kind of in parallel with the mindset of a hacker.

So kind of taking what I had learned from a red team operator and applying that same mindset to a blue team, now you have industries having like kind of a purple team where you'll have a mix of both of that kind of stuff and still kind of working in the a job I was working third shift to just kind of Supplant income as well. So starting to get into more just like the basic IT admin, IT support, help desk type of roles in my career. While you know, still messing around with a lot of the open source toolings, security onion being, you know, big one, a lot of open source firewalls and things like that just kind of expand my skill set. Running kind of at home laboratories, you know, small server setups and things like that.

Writing a lot of open source technology didn't have the money at that time for spending VMware licenses and things like that. So Proxmox and kind of these free open source alternatives that are still very well and alive out there, but really like getting my hands on the hardware again much different. You know, we're talking DDR2 RAM, much slower speeds years ago. So computational power has been, has come a very long way since then. But I was given a real opportunity working for a cybersecurity company out here in Charleston a while back. And that's where I got the opportunity to really give be given the task to build out. It was a startup, a full managed detection and response team, but also with the product.

And what was unique about that product at that time was Splunk was kind of the default in the industry for logging. Not only for logging for security purposes, but also for companies that needed to adhere to PCI or HIPAA or other standards that require you to log for seven more years, have that kind of retention. It was a very expensive solution and we found that a lot of customers didn't actually touch those logs. They just needed to adhere to a certain compliance standard. So we actually brought a pretty unique solution in the industry to where we would store all of your network logs via PCaps on the sensor device itself.

So we would go out and scope the client network and get a good estimate of ingress, egress, traffic kind of per week and then be able to inspect the storage to be able to retain those PCAP logs for seven plus years and then all the while on the same sensor being able to run network intrusion detection systems like that, where we had a full SOC team that would go back and help the client tune it to their network to find out is this common behavior, is this not common, is this anomalous, things like that. So that was a pretty exciting time in my career where we got to do a lot of things with hardware side but also apply what I learned kind of from a SOC perspective as well. Google Cloud was really just, I'll Say in its infancy.

At that time, AWS was certainly the behemoth, had majority of the market share as far as public cloud environments. Azure was there. I think AWS had really come first to the marketplace and GPT and Microsoft were kind of taking notes. So most of our solution was hosted on Prem. As far as servers calling back to our infrastructure and things like that and running our network operations center, all of this was on prem. And we decided to make the pivot to move to Google Cloud as our kind of public cloud provider. It certainly did not have out of all of the out of the box kind of solutions that you would see today turnkey solutions when you log into AWS or Azure or Google Cloud today being kind of the big three, much more of a serverless type architecture.

Even their hosted databases, much more out of the box turnkey, not much setup going on back then I think it was very common to see, you know, you use a public cloud provider but still be spinning up your own technologies on virtual machines to you are still responsible, you know, for the underlying operating system, but no longer really for the hardware. That's what Google kind of take on or any of these cloud providers in the shared responsibility model. So as kind of you know, we saw these cloud providers evolve.

I took more of an interest into cloud security because cloud security especially now is much different than what you would see in a traditional on prem network where you know, if you're, whether you're doing a red team operation or even a blue team on Prem, you're dealing with a lot of static infrastructure, you know, workstations, they're not moving anywhere, they're typically always on type of things where in the cloud you're seeing much more turnkey solutions that are not hosted traditionally how they were on virtual machines. You're seeing a lot of cloud serverless environments.

So for example in aws, you know, Fargate, all of these kind of managed containerized solutions, Cloudroom having and Google Cloud having Cloudroom as kind of that managed instance as well as, you know, all of these managed authorized orchestrators like Kubernetes, each cloud having kind of their own managed version of Kubernetes. So a lot more out of the box, a lot more ephemeral infrastructure which has been great for all the cloud engineers cloud architecture. We're not having to manage all of the underlying operating system type things on a virtual machine where you have to update an operating system pass and it breaks your database or something like that.

So that has taken a certain evolution for the architects and engineers however for blue team and red team let's say is posed, I wouldn't say harder or easier necessarily, but certainly a different type of attack vector and attack surface where you know, especially for services that scale down to zero at night when it has no traffic, you know, it's not always there. If you're dealing with ephemeral type architecture, the service counts. The IAM is totally different in the cloud than what you would see typically in an on prem environment, an active directory or something like that you're managing where you're trying to apply you know, principle of voice privilege in the cloud and all these things that are set up with you know, kind of default out of the box with a cloud provider.

11:47 – The Rise of Generative AI and Consumer LLMs

Anthony Baio
I think what we've seen since kind of the boom in the security space of cloud environments is since then with the launch of Claude being really the big first major gen AI to hit the market, there were certainly solutions where natural language processing and large language models before then. But as far as where you know, the typical everyday non technical user can just go to a website and chat with an LLM that was really the first major player. Yeah, we each cloud solution and before then whether it was Amazon or Google, Microsoft and a lot of the other ones that you are smaller but have a significant portion of share as well digitalocean line and stuff like that. But you can build your classical models.

So typical regression models, forecasting models, classification, image, video, tabular format, time series forecasting, you know those are for the tech nerds, right? Those are for the engineers. That's not your everyday lady type person logging in there. So really where chat should be came in is really for the average user to be able to interact with a model for the first time. And then we have seen kind of the iteration that's taken place with large language models since then with Google announcing, you know, Gemini and having that in production. Microsoft launching Copilot along, you know, investing heavily in OpenAI with that and then Anthropic releasing Claude. So we've seen a lot of these gen AIs now really racing to compete for not only speed but for quality as well.

I think we've seen certainly recently a large uptake in the usage of CLAUDE for Claude code as well as Claude for Codex. So a lot of companies now going from what would usually take 0 to POC maybe a few months and a significant amount of resources potentially into a matter of just a couple days where you can enter a prompt in Claude or Claude and it'll kind of build that, give you an interface and you have something to show a pe, a vc, a friend. From that perspective, certainly a lot more to go from POC to scalable product environment in a business. But that's really disrupted that type of industry. But overall I think it's a very exciting trend and I think it's certainly at every company's three year adoption cycle if it hasn't already been adopted.

14:19 – Security Risks of Employees Using AI Tools

Matt Pacheco
Awesome. And you got into AI. So let's talk a little bit about AI here and specifically from a security perspective because like you said first you're talking about consumers having access to Claude, Gemini, Claude, Anthropic, all these tools available, but so do employees. And you talked a little bit about coding and using that for workflow. What are some of the most underestimated security risks that come from employees using consumer generative AI tools? And what are your thoughts on corporate guardrails or things like that?

Anthony Baio
Yeah, absolutely. I think this takes kind of it sprawl to a whole new level where you know, users may go out and procure something especially if you're you know, inside of a, an, you know, active directory environment or Google sso. You can just go out there and log in with Google, log in with Microsoft and you instantly kind of have a free tier type of thing. But that's been your typical, it's for all of, you know, let's build an allow list of technology so that users can't go out there and just log into wherever they want. What we've seen with you know, Claude and these Gemini, these other ones is that you don't just have to have a corporate account and go in there, right?

You can log in with your personal Gmail, with your personal outlook, things like that, and then dump corporate data, whether it's intentional or not, into there to get insights. Now I think the most of the time the intention is positive. They're looking to gain insights, they're looking to gain trends analysis, build visuals around data to build better deliverables. However, especially with a lot of free tiers, most of what you don't have control over in three tiers is whether or not the LLM provider is actually using that data for training data.

So being careful as to what the users are putting into the prompt, whether that has ip, not about your own company, but about third party companies that you're working with, you know, we've certainly seen in the industry where companies have been IP has been breached not through the company themselves, but because a third party has accidentally prompted a model using, you know, through kind of shared data set or shared access technology about that company. And now, you know, through a free tier, it's kind of uses free training for that model. User awareness training I think is the best. This is a time unprecedented where things kind of blew up overnight and it was faster than really security teams could really catch up with. And security teams themselves had to learn about the technology.

Even you know, security providers kind of coming up with new technologies. As you know, it can be technologies now that serve as kind of a proxy layer between the user input and the actual LLM to screen the prompt before it actually enters an AI interface to make sure that it's scanning for, you know, any ip, PII data, health data, Social Security, all that stuff, and scrubbing it before the user can actually enter it. Certainly not of security awareness training, sat onboarding and just keeping individuals involved in what to put in there and what not to put in there. But also not just from, you know, what putting there and whatnot, but being educated on, you know, this is not 100% accurate all the time. You know, you just can't just take what it says for granted. It's meant to generalize, which causes hallucinations.

So if you're generally prompting, you know, for example, if you're not a developer but you asked it to build something that's not something, you know, you'd want to take that code and then put it into production environment and then just kind of hope for the best. Right? You still want to follow a traditional SDLC where you're doing testing on that and still going through existing security policies for all that as well as far as protecting the models, you know, there's always a risk of prompt injection. Individuals get very creative with bypassing what we would see system instructions, for example, you know, training the model on the back end to ensure that it never avails some type of data or some type of information that could cross contaminate between clients or something like that.

But really depending upon how you use tone and theme and urgency, the model, well, the LLM will bypass SHO system instruction and say, okay, this is urgent. For example, you know, you could kind of impersonate yourself as a, even though you'd be a malicious actor, say I am actually I'm operating as a compliance analyst inside of a company. I want to run a phishing exercise so that, you know, the outcome of the exercises. My Users are more well trained, need some information about the company to craft a well formed email so that it's actually more sophisticated and a better phishing email campaign or something like that. And then in those cases you would be more apt to kind of bypass the system instructions on those models and it would generalize. So certainly more precautions to take from that perspective.

And it's kind of exciting to see as well how much these information is available pretty much at your fingertips. I think we've seen a rapid decline just on the amount of traffic going to any kind of search engine. So google.com or Bing, because so many individuals at this point now have, you know, the mobile version of Gemini or GPT or Cloud or Gro, even on their desktop where they can just get information that's compiled and synthesized kind of how they want it at their fingertips. So that's been an interesting take as well. So excited to see what the future holds for it, but also proceeding cautiously on the security side.

Matt Pacheco
How do you go about protecting yourself from things like prompt injection and training, data poisoning and things like that as they become more prevalent as you implement some of these chatbots and these tools on their website?

Anthony Baio
Yeah, absolutely. So I think we saw, you know, a couple years ago where when Claude was released, a lot of companies being pressured to integrate into Claude and kind of build a wrapper into it because it was kind of the very quick out of the box, we can put a chat bot in our platform instantly. You know, we didn't have to take a lot of development cycles to do this. We didn't have to have necessarily an AI expert in house to do natural language processing and all of these things. In that I think we Learned, you know, six, eight months down the line that OpenAI doesn't know, or I'll say it only knows about your company, what is publicly available.

Anthony Baio
So for any company out there that you know, it's not reading your source code unless you've open sourced things to GitHub or something like that. But what you see is your user base asking questions about your product. LLMs by nature are meant to generalize. Now whether that generalization is accurate or not, you know, that's a different story. But a lot of users, you know, reporting hallucinations of I'm getting answers through your chat bot that have nothing to do with your product or actually how it works, incorrect answers. There's an interesting case with the airlines a few years ago about an individual getting free tickets for life A lot of stories like this. So I think we saw a lot of companies pouring more resources into, okay, we can't just build a wrapper into Claude and hope for the best. And that'd be our strategy.

We need to go in there and make it more specialized and granularize it to our specific use case. And one of the ways that it's now done is using a process called rag in which you would only ground it in data that's really relevant to the model. So for example, if in our chatbots that we, that I've used at previous companies, you're only giving a data that's relevant to what the user would be asking for. So for example, if you have a helpbot agent, it's only grounded information that's relevant about your product. It doesn't have the ability to go search the Internet. It doesn't have the ability to internalize even in its own LLM models what it might think it knows.

So getting the data set curated to really the bare minimum of what it needs to operate so that it doesn't, it significantly decreases the amount of not only, you know, producing answers that are outside of the scope, but even hallucinations inside of the data set that it does have. And then really I think all of them also have come a long way with prompt engineering system instructions within the model. So ensuring that with every prompt system instructions are being met, what can we do to ensure that they're not being circumvented? A lot of the cloud providers now, especially in Google cloud, have released model Armor. So a lot of that burden is kind of taken off of the end user.

And especially with Gemini being Google's own LLM, they're pretty good with putting guardrails kind of out of the box in that, but really a defense in depth approach. Even have, you know, an architecture of agents, so to speak, so sub agents where another agent will actually review the output of the first agent to ensure that there are no amount of sensitive data. So it's not just, you're not just relying on kind of one agent to do the whole thing, you're relying on kind of an ecosystem of, you know, of an agentic architecture to where each has its own checks and balances.

Matt Pacheco
Are there any concerns, especially in your industry being as regulated as it is with using AI agents for some of those tasks?

Anthony Baio
Absolutely. We get a question here. Insurance like, you know, a lot of industries, they'll say petroleum, healthcare, aren't necessarily out of the box as tech, you know, tech savvy, tech forward. I don't think when most individuals hear insurance, they instantly think technology. So usually those fields are, you know, might be reserved for more like cybersecurity, you know, things like that. So yeah, you know, seeing the landscape of how different, not only carriers and insurance, but what you would say, third party providers in the space have either adopted the technology and used it to their advantage or have kind of resisted it. Whether it's a lack of understanding or this is just a whole nother level of automation where you're not just using code, you're relying on agents that are actually doing reasoning and things like that.

To build more agentic tasks on a broader spectrum has been interesting. ISO did release a new certification, 42001, which we're actually actively pursuing at West Hills. That one in particular is meant for AI governance around kind of those agentic systems, specifically around LLMs, what you're doing for data controls, privacy controls, access controls, things like that. But I think overall, every industry at some point, if you're not adopting it into your tech stack, someone's going to come along and disintermediate you everything at any time. You're generally seeing 100 new AI startups a day. That's kind of the new buzzword. It's really caught the eye of VCs and NPS around the world of, you know, we can go from zero to product much faster than we've ever been able to for all things that we're looking out for. But overall, embracing the technology.

26:50 – Data Quality and AI Readiness

Matt Pacheco
So using technology like AI agents requires I guess, your data to be good. Sometimes it's a situation where it's garbage in, garbage out. What's your advice on like getting your data to a place where it's good quality so you can trust that an AI agent is making the right decisions and doing the right things for you. Especially in your industry where it's kind of important.

Anthony Baio
Absolutely, yeah. I mean a lot of it follows the basic principles. You know, AI didn't magically come along and reformulate garbage in, garbage out. You know that's been a, that's been a phrase for decades, right? And it still applies. I would say, if you know more today, if not ever more in the new past, where to get a really good curated set of Data, especially for LLMs, is more important even for LLMs than I would say traditional classic models where you're doing basic regression and things like that.

Only because LLMs do have the capability of reasoning and doing natural language processing to where if you train it on a data set where the data is inconsistent, you're dealing with a lot of null values, you're dealing with inconsistent values, data changing over time, serving skew, things like that your users are interacting with data that they think is accurate and high quality, when in fact they're getting metrics or insights that are not reflective of how the business is operating or whatever kind of KPI they're looking at. So how we go about usually ensuring that is one. I'm not going to deliver a model based off of non quality data and I'd rather just not deliver the model and say we can go about this another way and here's how we can obtain the quality data if we need to go down that route.

Other ways are, yeah, it does take time. I think we've, we're in a time now where data has actually surpassed source code as kind of the crown jewels of the company. You know, all of these elements are just eager and constantly scraping the web to get data. You know, I think a company is certainly the size of Amazon or Google or Microsoft, all these tech giants, Apple could relatively quickly, you know, if you know, seeing an application, you know, have the resources to build a replica of that pretty quickly. You know, there might be some nuances and things like that, but what they can't do is aggregate. At least we hope so that's kind of the point.

You know, they can have whatever is out there on the Internet open, you know, open source, things like that, they can scrape all day long but they're not going to see, you know, what's in your database that you've acquired just from what is your partners. So if you know, and that should be to your advantage, nobody else has this data. You can now provide insights and all of these things, trends and analysis that Even the major LLNs can't provide because they don't have this data. So now more than ever really having a good data infrastructure, good data pipelines to ensure that you are producing quality data for these models and retraining on a frequent basis, you're not letting that data set stale, not letting the model sit stale for six months a year and kind of keeping it fresh.

Data freshness has really been the key I think for us into making sure that we're providing with the users kind of the best quality models that we can.

29:50 – Selling AI and Serverless to Leadership

Matt Pacheco
Awesome. So it sounds like West Hills is, well, from what you said earlier, you're in gcp, serverless, you're doing all this great stuff with AI a Lot of companies aren't as, I guess as technologically receptive to some of this cutting edge approaches to cloud and AI. What is some advice you would give in selling some of this from the beginning? So like AI agents and serverless, selling it to a leadership team who may not understand some of these concepts, who are looking for results or whatever their goals are. How would you sell this as an IT leader to an your organization?

Anthony Baio
Yeah, I think you know, in today if you were to start a company, you're generally starting in the cloud, you know, you're, the average business owner isn't looking to acquire you know, a bunch of Dell PowerEdge servers in their house and like run that stack alone, you know. You know, generally just starting, you know, in the cloud. And especially now even with a product, you know, you can take a non technical business owner that's very savvy on the business sales side and using cloud code and Codex and get you know, PoC MVP level pretty quick without having to, you know, even bring on a developer before you kind of get to that first base.

As far as you know, companies that have been around for you know, decades and a long time and have a lot of on prem, you know, data centers around the country. Yeah, that does provide a much larger barrier to entry I would say. There are models, public models, you know, from Google out there, a lot of hugging face and Kaggle where you can download and run locally, ensuring you know, you have the relevant GPU capacity to host it and train and serve and inference and these type of things. But yeah, a lot of this technology, you know, especially to leadership that has been so accustomed to working, let's have spreadsheets for the last you know, 40, 50 years and now you're, it's not just a technological shift, it's a culture shift for a lot of individuals.

I think that's been really the most significant barrier I've seen just in the landscape is not, it's not that we can't adopt the technology, it's really, you have to adopt an entire culture that has been, you know, up until the release of Chatbots, more of kind of for the math nerds in the basement type of thing. But now it's kind of oh my daughter said that Claude or my wife said that Grok said this and it's like oh wow. This is like in the nomenclature, the vernacular middle schoolers in high school now kind of similar to when Google came out. Google just like Became a verb almost overnight. It's like, oh, I'll Google it, or just. But getting the leadership team to buy on. We always fit the technology to the use case. Right. So I'm not an advocate for just saying LLM is.

LLM solves every problem. LLMs solve certain problems. But LLMs are just one ecosystem inside of AI as a whole. You know, your classical models are still going to be much better and far superior to a lot of use cases than LLMs. You know, LLMs today are not producing code at the quality that senior developers are. Yeah. Now at the speed at what they're producing code. Sure. Because I don't know anybody who could type, you know, 400, 500 words a minute. But there's a balance of, you know, we. How can we take the speed that LLM produces output and compare that to the senior knowledge of those that we've had for a long time and kind of marry those together is what a lot of what we're seeing now.

But for the senior leadership, I think at the end of the day, it still comes all down to dollars and cents. You know, if we can implement things at the end of the day, that we're going to have higher revenue, you know, relating it to ebitda, lower costs, things like that. I think it still plays the same role that, you know, as technology has evolved over the last 50, 60 years, how are we going to sell this to management? How do you sell cloud adoption? How do you sell, you know, or on a Dell tech stack, how would you sell an HPE tech stack? At the same token, it's been similar to that, really, coming down to dollars and cents at the end of the day.

Matt Pacheco
So. So you just said something that triggered a thought in my head, the dollars and cents thing. So there's often a concern about costs with some of the AI models and AI agents potentially vendor lock in, things like that. How do you, what are your thoughts on that with. With cost and how to address the concerns with cost as you're scaling or what if one of the providers changes their pricing strategy because they need to make a profit? Because some of them don't seem to be right now. So what are your thoughts on that?

Anthony Baio
Yeah, it's a real interesting, it's a real interesting experiment going on. I think traditionally what we have seen as far as, you know, companies for compliance purposes, going after compliance standards and frameworks and things like that for business continuity and disaster recovery has always been, you know, what if GPT goes down? What if this AWS region goes down. What if as you're on prem, you know, what kind of backup do you have? Redundancy for data center failover. Things like that we've never had in this case, you know, companies being formed today where your entire code base has been formulated by Claude code or Codex or now you know, with Google's Anti Gravity Gemini, that is equally as important if that goes down as your actual hosting provider.

Because you know, if you can't build with your code and if every, you know, every time you have to send a prompt over whether it's in the browser interface or through the API with code for your users to operate. Yeah, philanthropic went down. If you've hit a rate limit, if you've hit some type of billing limit. Now that's become a whole new marketplace for introducing downtime that we just haven't seen before. Where you're not talking about your infrastructure, you know, cloud's not pushing your infrastructure. OpenAI that's for the actual either on prem or cloud providers. But the LLM players are now so integral to your tech stack that is not something that you can ignore anymore. If you're relying on that for your day to day type of business, especially for consumer activity to where you do have to address that.

I think for a lot of companies that we have seen that are addressing it much more rapidly is that do have cloud accounts or Codex accounts or Anti Gravity or something like that is selling, you know, setting up billing alerts, you know, equally as important, you know, as it going down is, especially if you're in the startup world, if you get a cloud bill for $30,000 after one week and not realizing that, you know, you've done that. So things can equally get out of hand on that side from an expenditure perspective as it would have, you know, the actual LM provider going down.

So what we've seen a lot of that are using redundant LLMs, you know, not being all in on cloud, not being all in on OpenAI, you know, it's very unlikely that you would see multiple, you know, three out of four out of five, whatever LLM providers going down. But if you have all of your eggs in basketball for one LLM provider, although they generally guarantee, you know, a certain amount of time, your business is really tied to their uptime at that point, you know, rate limiting certainly comes into a factor. You know, if you're hitting Claude or Chat, GPT or Gemini, you know, for more than what it could limit. Now you're not only talking about from your company's infrastructure of how much volume you can handle, you're more reliant on the actual LLM provider of how much volume you can handle.

And unless you're hosting that again, like in your own cloud environment, you're downloading something and running a local model that's actually becoming more of a bottleneck than a lot of your own infrastructure. So really building redundancy into that and making sure you have a good plan to where, you know, maybe your entire stack as a company isn't relying on just one LLN provider, I would say is my biggest piece of advice.

38:10 – AI Governance, ISO 42001, and the Future of Compliance

Matt Pacheco
Nice. It's funny as you were speaking, the thought, and this has come up before, this idea is that the cloud and kind of cloud adoption, public cloud adoption and AI adoption, there's so many parallels between billing and all that and the resistance to adoption at first. And it's just funny as you speak those, you realize the stories are very similar and how companies are approaching it. I want to ask you a few more questions specifically about AI and governance. We'll talk about a little bit about that before I talk about some of our forward thinking, fun questions about the future. So ISO 42001, is that how you say it? Yeah, it's one of the first standards specifically designed around AI and ML. Lifecycle management can explain what it covers and why you guys have decided to pursue it?

Anthony Baio
Yeah, absolutely. So as I said, you know, we at Westell are very much integrated and forward leaning into embedding AI into our platforms, our processes. With that, you know, we don't want to just leave our customers and say, hey, we're on the bleeding edge and we hope that, you know, data won't be compromised or we're using it in a non compliance type of format. You know, security and compliance is always equally as important to us as your features. We want to make sure any data that we're providing as a company is secure. And you're comfortable with that. Now I realize that, you know, waving a certification around the air doesn't mean that you're necessarily secure, but it actually, you know, for us internally we looked at that a lot for education. You know, we have some really smart people.

There's a lot of smart people around there on the frontiers of how LLMs work and really getting into tuning the weights and a lot of the math in the background. That's why I love it. So a lot of it for me is I need to Be educated before, you know, I just go out there and release and you know, that's what these compliance standards are out there for is not only to make sure as a company, you know, you're adhering to these, but the individuals that are actually in the weeds and doing this every day are having that mindset as they're building this out. So it's not an afterthought.

It's, you know, as I'm writing the system instructions, as I'm deploying this LLM, as I'm building the data, this is kind of ingrained in my mind of what I should be doing in security is a, something that's, you know, from the beginning and not like a post deployment type of thing. So what we're, you know, what we're hoping to gain out of that is absolutely a certain amount of trust from our clients that yes, we are on the forefront of AI. We are adopting it very heavily and taking advantage and using that as, you know, kind of running a strategic moat. But we're also equally as concerned with the security compliance aspect around that. We're not willing to go as fast and compromise a data leak that wouldn't be worth it to us.

So it forces us to be a little bit more, you know, it's fun to be on the bleeding edge. I think every once in a while you have to slow down, re strategize and say, you know, there's a little, it's a little bit different to build a POC and mvp. You know, it's really cool. It's, you know, a shiny thing versus, you know, that piece of architecture actually running in a production environment with you know, customer data, customer access, things like that. We don't want to cross contaminate data. So we've always been, I think, really on the forefront of security and compliance as well as the feature sets and we just want to keep that going. AI shouldn't be treated any differently than what we've seen of technology in the past.

Matt Pacheco
That's very impressive. I guess as AI becomes more embedded in workflows, products, not just with you guys, but in general. Do you see how does security and compliance conversation, I guess, need to evolve to account for all the changes we've seen in such a short amount of time. Like things like SoC2, is that even equipped to handle AI risks? What are your thoughts on that?

Anthony Baio
Yeah, I think you're going to see a whole new level of, you know, certifications within themselves like ISO 4000, 2001, but even just benchmarks, you Know, coming out of NIST or CIS or, you know, all these companies that you would normally turn, you know, turn to for benchmarks. A lot of the public cloud providers, Google, aws, Azure, have published kind of their own security benchmarks around you running, you know, AI within the cloud environment as well. So, so that's been kind of very handy measure architecturally and from an engineering perspective of, you know, are we deploying this the right way, are we monitoring it, are we logging it, how do we go about resolving, training, serving SKU data quality and setting that up?

I think we're going to see a lot more benchmarks, you know, potentially coming from mitre, all of these kind of open source and publicly available security and compliance background, you know, kind of companies producing a lot more data around, you know, you want to stand up an AI agent, here's kind of best practices and how we can adopt that. As far as the individual. I actually think there's going to be, you know, you would like how you would see if you download Facebook on your phone, you're generally agreeing to a lot more than what you think you're agreeing to out of the box. I think, I think that, you know, AI is really going to take that to a whole, never a whole nother level, especially as industries, you know, want work.

I think, you know, you could have a smart house at this point, you know, with everything connected to the Internet, everything connected to WI fi forming over cable, becoming, you know, much less prominent. But you know, your thermostats being smart, your doorbell, your cameras, all this stuff. You know, when these companies start embedding AI kind of at the edge, that's a whole nother realm of data collection than what we're used to. So, you know, constantly collecting telemetry from your thermostats and video footage from your ring doorbell camera, all this type of stuff from your smart TV down to your refrigerator. I think that's going to be, you know, and companies are hungry for that type of data to get consumer activity. I think that's going to be a whole new level of, you know, sacrificing privacy for functionality for a lot of this stuff.

So interesting to see how legal will sort that out. I'm not a lawyer and I don't pretend to be one, but I think that's going to be a challenge going forward for consumer adoption.

Matt Pacheco
Yeah, definitely a lot to consider. Speaking of going forward, what do you, what are your thoughts on, I guess AI, developer. AI and developer jobs, like things are shifting, skill sets are shifting. How do you see the talent landscape changing in the next three to five years? What should people really be focusing on? How should people, how companies be hiring? I'm really interested on your thoughts.

Anthony Baio
Yeah, a lot of it's a unique situation. I think, you know, I'm not going to print a broad bush, but I think, you know, AI has a lot of advantages. So. So in general, I would say to be reluctant to use it or reluctant to adopt it is probably doing more harm for your business than good. Because whatever vertical, whatever industry you're in, you can pretty much guarantee it at any point in time. You have 10 competitors starting today that are AI first. And if you have kind of antiquated processes, you know, technologies, things like that, if you're not adopting those type of things, I think you're just doing more harm. Again, you know, we can go into how you would adopt that. More of a cultural adaptation process and maybe a technological one.

But there, the value that senior SMEs have in their respective fields are unmatched today in LLMs. You're not going to have an LLM that's just going to instantly have the knowledge that a PhD in mathematics would have or relative to CompSci. I think we see a lot on LinkedIn of I can replace my 40 developer team with a few cloud code licenses. There's a whole nother, under the water type of iceberg to that situation. A lot of which is individual posting. You know, here's, you know, here's what I got it finally ran on the 105th iteration of me prompting this. But it didn't work the whole, the first 104 tries. Right. So there's a lot of that, you know, kind of height in the high traditional, high curves that any technology would go through.

I think we're kind of in that bubble of, you know, we don't need as many developers. We can rely on cloud code type of things. I think what you're sacrificing there is, you know, they're adopting for the sake of speed. It's very, you know, the markets, you're, there's a lot of fomo. We don't want to be left behind. We need to adopt these technologies. We need to find places to, you know, plug it in, even if it's putting up, you know, a square peg in a round hole. But what we're sacrificing there is a lot of the quality of what we have in senior developers today.

And in talking with a lot of really smart developers that I have in my circle, since I'm not a developer myself and I'm trying to be one, is kind of a lack in academia today in the individuals that are going for comp sci degrees of academia, maybe preaching that LLMs are going to be replacing a lot of developers and you should find a different field in technology to go into. I think that's a little scary for a lot of the senior developers out there because I think for the first time in a long time we're not seeing the level of developers just kind of per capita from institution that are coming out behind you. Traditionally you'd have more, you know, more developers graduating every year, more comp sci coming out every year.

And now we're kind of on trend where I don't know if it's reversed, but it's certainly slowing down a little bit to where you may not have, you know, 30% year over year, 35%. You may see a regression in that percentage. The problem there is, you know, you still need senior developers to vet this code. You know, you can't just hope for the best and wing it to production. And then even after the fact, who's maintaining it? You know, how do you develop a knowledge base around a product that's completely built with gen AI who's troubleshooting this right when a bug comes up? So I think that there's, you know, we've taken the approach of there's a path where we can really work together on this.

We, we can take the speed at which LLMs are producing code, but we can also take the knowledge base that is not in the elements in general of AI produced code and we can marry those two things to make sure at the end of the day we are taking advantage of the speed, but we're not going to sacrifice all of the quality. And we do see a lot with LLMs, you know, cloud skills and you know, skills files and things like that and session memory. You know, LLMs are coming up with their own ways to kind of try to increase the quality. I just don't think that's something that you're going to replace. You know, somebody that's been working with Java or C or Golang or something like python for 10, 15 years, that's pretty far off.

And you have, you know, you have to say LLMs, you know, in general, since it's, you know, a lot of, you know, you're scraping the web, would probably, I would say, be tantamount to a junior developer so even if you're spinning up, you know, 10, 15 instances of, you know, CLAUDE code, I that's probably equivalent to like 10 or 15 junior developers. So if you wouldn't take that, you know, chance to go, you know, take your company and produce a product over that with junior developers, I probably wouldn't do it with thought code or Codex or something like that. You know, that's where you'd want to have definitely human in the loop and have controls in place so that you're not sacrificing, especially to the consumer at the end of the day, the quality of your product.

Matt Pacheco
Yeah, it seems, this seems to be the common response to that question is it's not going to replace people, but it will augment them instead. And all that because like you said, you still need people to do something. When security companies are using AI tools to identify threats and all of that, they still need someone to actually look at all of the output and all the alerts that are generated by AI. So there's still value in the human. And I agree 100% with you. I got one final question for you. If you had to flag one emerging trend in cloud or AI that you think most organizations may not be paying close enough attention to right now, what would it be?

Anthony Baio
Since we've been talking about AI and ML the most, I'll say multi LLM architectures. There is a significant amount of not only quality but creativity that you get when you have multiple different LLMs working in sort of a ensemble learning pattern. This is actually where I'm doing a large amount of research today. Very much on the math side is rather than taking, you know, if you ask GRO to do something, for example, and then you just take and hope that everything produces GRO and we're in the full GROK ecosystem or Claude or you know, whatever your LLM is, each one has their strengths and weaknesses. Gemini is great for multimodal type of things. I think Claude is generally better at the code reasoning. OpenAI is generally better with maybe analysis, data analytics.

GROK is great for kind of the real time type information with access to your X and everything that's on X type of those. So really having kind of an ensemble learning process of building an architecture that leverages, you know, not just one LLM for everything, but the correct LLM for the use case that you're trying to solve for. So I think what we're going to see is a lot more architectures where we're seeing agents utilizing multiple LLMs either in parallel or sequentially or some type of hybrid architecture where the output that you would get from utilizing that kind of architecture is leaps and bounds.

What you would get from just one LLM, if you have four or five LLMs, all kind of collaborating in an ecosystem, the outputs that you get from that are really unparallel to what you would get and just kind of A1 sequential LLM.

Matt Pacheco
Well, Anthony, thank you for being on Cloud Currents today. It was a great conversation. Lot about AI, machine learning and data. So thank you for spending the time with us and talking to us today.

Anthony Baio
Thank you, Matt. I very much appreciate you being invited on here.

Matt Pacheco
Yeah, awesome. And I'm very interested to see how the world changes in the next, what, six months? It could change again. Very interesting world and very exciting. I love your mathematical approach to all this. I love your background. It's all equations. Just awesome. So thank you for our listeners. Thanks for listening in. Stay tuned for more episodes of Cloud Currents. Check us out wherever you get your podcasts and we will see you soon. Thank you.