Guardians of the State: An Air-Gapped AI Fortress for Consumer Data — Rachna Srivastava, DFPI
https://www.youtube.com/watch?v=2WZsT-znFTQ
AI Engineer · 21 min · 166 sentences · sentence-level transcript from YouTube's caption track; each line begins at the time shown

[00:01] [music]
[00:12] By the time this presentation ends, someone, somewhere will make life-altering decisions based on something generated entirely by AI.
[00:30] Last 30 years, digital infrastructure is based on these unwritten rules.
[00:39] If you see a face or you hear a voice, you trust someone behind it.
[00:47] If you see a signature, you trust someone has approved it.
[00:53] Every business transaction, every government workflow is based on this foundation of trust.
[01:03] So, trust is a invisible layer on top of it all this digital infrastructure based on.
[01:12] But, generative AI trashed it completely.
[01:17] AI agent can clone a voice.
[01:20] Synthetic face can bypass identity check.
[01:25] AI can impersonate people at a speed no criminal organization have been able to do that before.
[01:36] So, in the era of generative AI, seeing is no longer believing, neither is hearing.
[01:44] The cost of deception is collapsed and the speed of deception is exploded.
[01:54] Hi, my My is Rachna Srivastava.
[01:56] I work for California Department of Financial Protection and Innovation.
[02:02] Our mission is to protect 39 million Californian and their financial identities.
[02:11] Let me tell you what we do.
[02:13] When a massive fraud attack in in the state of California, we identify the fraud, we examine the fraud, and we run enforcement.
[02:26] Let me take you behind the scene what we actually do.
[02:29] Fraud comes, we investigate the fraud.
[02:33] When the investigation happen, we collect all the evidences for the fraud.
[02:38] When we have all the evidence, we take that evidence to the court.
[02:43] And in the court, the defense attorney has only one job, to attack the system that we have built.
[02:54] So, the question arises, how should we build a system which is credible?
[03:03] What are the attributes we need to have in a system which can appear in the court?
[03:11] And appear in the court means system should be defendable.
[03:16] Defendable means it should be we should be able to explain the system at every step of the process.
[03:23] We should be able to reproduce the issue at every step of the process.
[03:27] We should be able to audit the issue at every step of the process because everything, every data that we produce, it's going to appear in the court.
[03:40] So, how do you build that kind of system?
[03:43] You Since we deal with financial data, we have to ensure that the data is secure for sure.
[03:51] We have to ensure that the data is evolving and learning.
[03:57] Why?
[03:57] Because the fraud space is changing every day.
[04:01] We have to ensure that the data is available to be appear in the court and reproducible.
[04:10] So, what I'm going to show you now, the system that we built, and the lessons we have learned the hard way.
[04:21] So, the first and foremost decision that we have taken was to build our solution offline.
[04:31] So, you might be wondering like it's it's extreme.
[04:35] Like, why are you Okay, why do you want to create a solution offline when we have a secure cloud?
[04:44] So, before we show you what we build, let's talk about what do industry do in the normal way.
[04:54] So, the gold standard of security is encryption.
[04:59] Encryption means you encrypt the data at rest or encrypt the data on the transit.
[05:06] That means the thing is but for machine learning model to work, the data has to be decrypted and available in the plain text in the model memory.
[05:21] And if the data is already decrypted and present in the plain text, it is prone to prompt injection attack.
[05:30] Secondly, let's talk about private endpoint.
[05:33] Private endpoints are actually a isolated environment given to you from your cloud provider.
[05:43] But cloud providers actually own the on the on the disk.
[05:51] So, the question the thing is that according to the Cloud Act federal government can access your data which is in the cloud and they don't even have to tell you that your data have been accessed.
[06:11] Let's talk about certification.
[06:14] FedRAMP compliant, I also certified SOC 2 compliant.
[06:20] All these compliance are just paper.
[06:24] And we have seen these highly compliant organization fail over and over again.
[06:35] So, only way you can make your data secure to be available in the court to present as a as a evidence is to build the solution offline.
[06:53] So, like everyone else we thought like what a big deal, let's just download a open-source model, create an isolated environment, spin up some GPU, add some system prompt, and add some guardrails,
[07:07] and done we are running.
[07:09] Push the live data into it.
[07:12] We did the same like everyone else.
[07:15] And the model the system collapsed in 2 hours.
[07:22] And our first instinct was you know, we took the model free model from online.
[07:29] Model is not good for our use case.
[07:35] But the issue was we were we were treating machine learning model as a magic box instead of as a data pipeline.
[07:49] So, anything that garbage comes into the system, we were expecting the model to clean up and do all the processing.
[07:58] So, to solve this problem, we introduce Kafka for data ingestion, a Spark for data processing, and large language model for reasoning.
[08:10] Three different tools to solve three different problems.
[08:18] We use Kafka for these three things.
[08:22] First, the data that enter into our system does not come at a consistent speed.
[08:30] When a fraud attack happen, it attack all over the state.
[08:35] So, we get a high spike of traffic into our system.
[08:40] So, we needed a tool that can buffer that traffic so that the downstream component can access the data at a consistent speed.
[08:51] And Kafka did that.
[08:53] Secondly, we wanted all the events that occur in the system to come and store in a sequential order.
[09:05] When did you open the account?
[09:06] When did the transaction happen?
[09:08] The order of events in the fraud is extremely important.
[09:13] Kafka helped us solve that problem.
[09:16] But the main problem that Kafka helped us solve is reproducibility.
[09:24] In Kafka, it let you move the checkpoint to the point at which the decision was made, and it helped you replay the event.
[09:34] And this replayability of the event is the proof that we present in the courtroom.
[09:44] So, now Kafka did solve our data storage problem.
[09:49] But, the data that enter into the system is really messy.
[09:54] We get 10 different format of bank statement.
[09:57] We get audio file.
[09:59] We get screenshot, facts statement.
[10:05] And, if you dump all this data to model memory, then no wonder model hallucinate.
[10:13] So, we introduced a Spark because a Spark let us process this data behind the scene in the clusters of CPU instead of working on GPU.
[10:27] And, Spark let it let us clean the data.
[10:32] And, then when we process send the clean data through the model memory, to our utter surprise, the same model started giving us so good results.
[10:45] It found the connection between the points which we never thought expected.
[10:52] So, we learn our first lesson.
[10:55] The lesson is most of the data problem in AI are data engineering problem wearing AI mask.
[11:06] So, I recommend you to consider using data engineering tool to solve those problems.
[11:14] Now, our model memory is filled with a very condensed clean data.
[11:22] But, then we found our next issue.
[11:24] The issue was the memory was filled with lot of sensitive information.
[11:30] Your credit card number, your bank account number, your social security number.
[11:37] So, we introduce something called cryptographic vault.
[11:42] Which is basically a SHA-256 cryptographic algorithm with hardware security module.
[11:51] So, in the simple sense, it means that as we get the data into the system, cryptographic vault which convert the data into cryptographic hash.
[12:03] But, the key that is used to do the conversion is physically attached to our server rack.
[12:13] So, if tomorrow somebody is able to get our data, they have to walk into our office physically, break the server rack, get the key to actually make sense of the data.
[12:28] Then, we learn our second lesson.
[12:30] If the stakes are really high, trust hardware over software.
[12:41] Now, we have a model which is working really well.
[12:45] All the PII is redacted.
[12:48] We thought, let's just run a quick round of load testing and release this product.
[12:54] The problem is in the cloud, the scaling is unlimited.
[13:01] You get a high spike of traffic, it is spins off more server, it's it's distribute the load to those server, and you are done.
[13:10] But, when you are creating an isolated environment, you have limited GPU, limited compute, limited VRAM.
[13:19] So, we have to step back and figure out what is the actual problem here.
[13:25] The problem was we were using one state of the art machine learning model to do every single processing.
[13:35] The same model was doing the summarization, entity extraction, fraud ring detection.
[13:46] So, we were actually making a neurosurgeon take the blood pressure of every single patient.
[13:55] To solve this, we introduced triage nurse or semantic router.
[14:02] And the goal of semantic router is as we get the data, it analyzes the data and forward the request to the smallest possible model which is capable of processing
[14:16] that request.
[14:19] And by making this simple architecture change, we found that more than 80% of of our task could be easily done by the smallest, fastest, cheapest model.
[14:36] By not adding any new GPU, we could process three times more traffic and cost of processing each request reduced to nearly 70%.
[14:55] Now, we have done the unit testing, we have done the load testing, and then we found the hardest problem of all.
[15:08] The problem was we built this highly secure system, but the system was not learning.
[15:19] System did not know what is happening in the threat space.
[15:24] So, the question arises, how do we make our system learn without creating a security hole?
[15:36] Usually in industry, people solve this problem by configuring software firewall.
[15:44] But any configuration can be misconfigured.
[15:48] And once you have misconfigured, your highly secure system will be highly exploited one.
[15:57] We decided to not trust the configuration, and we took help from physics.
[16:04] We introduce one-way data diode, which is basically a fiber optics cable physically cut into half.
[16:16] The first half of the cable is connected to the internet to receive the data.
[16:22] Second half of the cable is connected to our solution.
[16:26] First of the half of the cable has laser transmitter, which receives the data from the internet and transmit the data to the second half.
[16:36] [snorts]
[16:36] Second half has a laser receiver to receive the data from the first half.
[16:41] But there is no laser transmitter from our end to the outside world.
[16:48] So, it is physically im- possible for data to leak from the system.
[16:55] And this is how we ensure 100% guarantee of the security of the system.
[17:09] Data diode solves the problem of directionality of the data.
[17:15] But anything enter into the data is is considered to be unsafe till proven.
[17:23] So anything from outside first first lands into quarantine zone.
[17:30] Where we run a spark job that runs a validation on each input data.
[17:36] And when all the validation is successful, then the data goes to the production layer.
[17:45] In the production layer, we also save the data into Apache Iceberg.
[17:52] Apache Iceberg is a time traveled queryable immutable data store.
[18:02] We enter the data in this data store because 2 years from now one of our results from our system goes to the court.
[18:15] We cannot present We cannot go to the court and tell them, "You know what?
[18:18] This system is result is produced by AI and we don't know anything about it."
[18:26] At that moment, we travel the Apache Iceberg, go to the point at which the decision was made and get the state of the system at that moment from the database.
[18:41] And that is state of the system appear as a proof in the court.
[18:49] And that is how we build a solution which is secure, which is evolving, and which is defensible in the court.
[19:07] This is the whole architecture end to end.
[19:11] If you see this only at the very end of the system, we are when a user logs into the system, they we authorize the user into multi-factor factor authorize
[19:25] authentication and only then the data is reverted.
[19:30] Data is not opened till the very end at the browser where the user is actually evaluating the threat case.
[19:45] So, don't look at the solution as a fraud detection solution.
[19:49] This is the architecture of the future.
[19:53] Soon the same architecture will be used to predict the health care, banking e-statements, legal and other domains.
[20:04] In the end, I just want to say one thing.
[20:08] Build solution that can be trusted.
[20:13] And remember, trust is not a policy.
[20:17] Trust is a physical property of the system.
[20:22] You have to build the trust from day one into your hardware, into your physics, into your architecture or it's not there.
[20:33] It's that simple.
[20:35] So, years from now nobody will remember the models you trained.
[20:38] Nobody will remember the benchmark you received.
[20:42] People will only remember the solution that you have built can be trusted when it needed the most.
[20:53] Thank you so much.
[21:10] [music]
[21:12] Mhm.