Transcription
All right, welcome everyone. I'm Andrea. I'm senior engineering manager here at Infinity and I'll be your host for today's global edip.
So, as usual, we have a pretty exciting agenda. So, we have a couple of uh pretty extensive updates both on the road map and caffin. Then for the picture part we hear about uh an on CNS which is onchain name service alternative to DNS and for the demos we have a couple of internal demos one on the component model from do a proof of cons of this and uh about immutable blob storage for cafe and for the community demo today we have the diode which is one of the first project in integrated our production system with be keys.
So without further ado, let's dive into it. And the first one is Sam with the road map update. Please help.
>> Thanks Andrea. Let's please go to the next slide. My name is Sam. I lead the engineering team here at Definity. So welcome ISP community. Hello our to our entire team. So every few months we do a bit of an update on where we stand with respect to our road maps. uh the last such update was just before the summer break that was in June and so even though there was summer break uh July uh now August 2 months later we have already more milestones we can report on so it's exciting to give a bit of an overview of that.
uh in the compute platform space uh there is one activity where we work on quite a bit that's this block storage I'll not talk too much about it because we have an extensive update today byan Dave later on in this global R&D um you may have also may remember some updates we shared about um out calls where you uh can configure more has more configuration options to not rely on full consensus that's maybe another important update we have been working on in the compute platform space.
in the decentralized AI space or just AI in general we had in June uh the event in Zurich about caffeine. Then in July there was a big event in San Francisco. We also reported on that. So that concluded this alpha phase and today uh Kepler will give us a bigger update than what's ahead. So that's why you find this little coffee mark but I'll be reporting in a bit about ignition which was the milestones about having LLM access in smart contracts. So that's something I'll talk about.
There was not new big milestones in chain fusion. We are working towards the Dogecoin integration uh in in privacy. We work uh as well on more more more um protocol changes after we have launched vet keys. So there's not so much new to report there. But in platform decentralization, we'll achieve the Levitron milestone. I'll be talking about this.
In the identity space, we launched a new version two of internet identity with a completely overhauled UX. There's still more changes coming. It's being integrated in caffeine and other new apps. So there that's a space that is very active these days. Digital access assets, we do some work for uh real world assets. Um that's the the main focus there right now. And then in the governance and toconomics space we achieved the neon milestone which I will also be talking about developer experience. Lots going on but not covered today. So that gives you a bit of the overview of all the different sort of swim lanes or areas we have in our road map.
And so that's why now let's jump in and look at decentralized AI. Here we uh declared this ignition milestone as complete. uh already back in February we had a presentation by Islam where he showed how you can access LLMs that are actually running offchain but you can access them quite conveniently from a smart contract in the meantime there's been more library extension you find Motoco libraries Rust libraries TypeScript libraries to make use of this LLM AI worker and that complements basically the ability to run AI models onchain but if they're too big or you really want one of the frontier models to be used then you can use this approach and uh we have received interest from the community and improved based upon that and we concluded now this work with the uh ignition milestone overall our focus in the AI space is very much on a on on caffeine going forward so we'll not spend too much more time on running onchain AI for now for the moment. So that's what we did in the AI space.
uh let's please go on to the next one. um you may remember early this year we achieved the solenoid milestone which was all about uh decentralizing the boundary nodes. Boundary nodes are the infrastructure that provides access to the internet computer and that gives users the ability to choose how they connect to the internet computer and so that that allows them to be independent of for example the definitity foundation. they can pick their preferred some sort of gateway. And based upon that, the next step now was to make access logs and metrics that give more information about canister methods being called or client identifiers, request types, all of this type of interesting information that allows you to to see what's going on on the IC available to the public. And um Rudy gave a comprehensive summary about that in July when this milestone was achieved and I included a picture of the forum post which shows you the architecture and the medium post that explains what was done. So everybody who would like to know more how the IC is used and and learn what patterns we can extract from that I recommend taking a look at that. So that's what we did in terms of platform decentralization and then please to the next slide in the governance and tokconomics space.
We were working on various sometimes even quite small improvements but they all can be sort of summarized under in activating the community making it easier to participate understanding how you stake so that you can participate in governance but also that you know how much yield you would get for the neurons you have staked and so that all this work was summarized under the neon milestone. And for example, we had a presentation in August by Yousef explaining updates to the NNS front end app which now indicates much more how much yield you would get or guides you better through the staking process, various UX related improvements and more on the back end. Uh we had two presentations by Alshabir on SNS uh liquidity pools. So that allows basically SNS's to put some of their treasury into pools into liquidity pools so that the community can more easily trade tokens. So there is more liquidity more fluidity for uh tokens and hopefully engages again the community. These are two examples but there was much more we did under the neon milestone and that's another milestone we achieved uh in the last few or actually it's not completely achieved but we are just about to achieve within the next few days. So that's the third update I wanted to share and with this we already go back to the next slide please where we see again our road map again today you hear more about blob storage you will hear more about caffeine and I'm sure you will hear also more about developer experience in the weeks to come because there's a lots a lot going on in that uh area so with this I hope that gives you an update and see that even though there was summer break we were quite busy uh back to you and thanks out and it's really nice to see that all things happening on restaurants and the next one is already the one that most people are waiting for and so Kepler please tell us about what's happening on caffeine.
All right. Hello everyone. My name is Kepler. I'm the engineering lead for Caffeine. And I'm excited to tell you more about Caffeine. You probably have heard already a lot about it. And uh today I actually before we jump in and talk about what will come next, I actually wanted to touch base a little bit of why caffeine and why now.
If you go to the next slide. So Bill Gates in the mid90s he said if your business is not on the internet then your business will be out of business. However, this still applies to these days. If you have an idea, if it doesn't fit, let's say um a very specific scenario that actually building a little bit more complex idea still required users to go find a team maybe find a company that will do that for them. They need to hire a designer. They need to hire a developer. And then after they have the application created, they still need to make it operationally ready, deploy it somewhere, make it run and make sure it keeps running.
If we go to the next slide, this you can imagine is very hard still. So for uh an end user or for somebody that's not technically savy, not an entrepreneur that actually will be able to do things end to end, they actually if they don't have a lot of funds, they won't be able to actually do that. So imagine that a user would come have an idea and they could actually just say with natural language that here's my idea. Maybe I want to build a portal. I want to build a gallery for my family. I want to build an e-commerce app where the checkout flow would look this and this away and I want to make it live on the internet and operationally safe potentially even in the same day. Well, that is what caffin brings to users and it's possible now because AI had some advancements in the ne last few years and we can make an operationally safe through the technology that ICP provides to users and it will also be very soon available for anyone with a smartphone. So that's the team that uh the person would hire that instead of hiring the team actually they just talk to AI and yeah then deploy it to ICP.
If you go to next slide you can then also see the journey we've have done so far. So on June 3rd we had the world computer summit where in just a few months we were iterating through different versions of caffeine. We then did a demo where would things were still uh in closed version only very certain people would have access to it and we opened up for invitation code. So this means that users would be able to then go and add their email and then they receive an email to get the invitation code to then access and try the platform. Then in San Francisco in July 15th we actually launched the alpha. So that's when the first users got access to caffeine and could try out and receive the invitation codes with in that same day we actually did some hackathon as well where we distributed some prizes users built in a few hours applications where they then demoed very interesting ideas and it's interesting to see in between July and now how much things have evolved. We now see for example some ver some users have actually generated version of the same application that they created upwards of 358 versions where it gives you an idea of how far they can go by talking to AI and adding extra features and making things more complex. We are now also in the middle of general availability rollout. What this means is that we're going to open access more broadly and soon we'll also be making available the app market for caffeine.
If you go to the next slide, we have divided the rollout in different phases. So phase one is the public master chat. So that happened uh in August 19 now where we opened the caffeine master chat to everybody. Well, I think it's still important to remind everybody that caffeine is not uh the chat CPT like application where we'll just go and chat with but it also provides that functionality. But the actual main feature that you will get with caffeine is of creating applications that you provide your build instructions or your ideas and it generates those applications and then deploys that. to do that. Users still require the invitation code and to you and they will get it more now also during this month but it is already available for anybody that lands in caffeine.ai to then talk to caffeine talk about self-writing internet talk about the internet computer and get some insights and for those that do that they can also add their email to then get an invitation code too.
If you go to the next slide, in September we will open phase two of uh and which starts soon uh next week. So we open phase two uh of the general availability which means we'll on board the full weight list of caffeine for those interested they can uh look in caffeine AI on X and we'll also post more updates there. But what this means is so far organically we already had 15,000 per people waiting in the wait list and from those we only invited around 3,000 at the moment and these users can go in once they get invite code to build applications as what you see here in the picture of for example I mean I made this uh for fun just creating an application where I can upload some photos for my uh vacation that I came back now from Portugal. And uh once this is available then all of these 15,000 users will then be able to go in caffeine and explore steel without any payment attached to it. They will be able to explore all the functionalities of caffeine.
Then we go into the third phase of the release. So if you go to the next slide which is the public uh availability. So really the general availability of caffeine and this happens in October where we can really welcome the self-writing internet and users will not need any invitation codes. So everybody will be able to go into caffeine. Everybody will have some free credits to try the platform and people will be able to then create some projects. They'll be able to iterate on some of the their ideas. But for those that really want to scale their ideas, then they will be able to then subscribe or have a pay as you go plan where they'll be able to then buy some credits and use the platform with its full functionality and all the all the ability it has.
Then on phase four is when we really unlock most of the functionality of caffeine where we create the we are the creators economy. So caffeine at the moment has this app market. We don't yet have uh a date to release this. This is exactly phase that comes after we go general availability. But the app market will enable uh any creator any builder in caffeine to after they have their application that they find it in a in a stable way that they consider live and that they consider they can for example resell it. they will then be able to publish it to the app market. They will be able to do it for free. They just make it available for other people to use or they can resell it. And if they do that, then you can imagine the possibilities of now anybody with an idea can turn this idea to life and then potentially make some revenue out of what they publish.
And then what does this mean for ICP? So if you go to the next slide. So caffeine will expose the internet computer to many more users. You can imagine now that all of these end users that are using caffeine, maybe for those that are not even aware of ICP, they'll become aware through caffeine and understand about where things are live, how it's operationally safe. And caffing can help educate users about importancy the importancy of data sovereignty and how important is on how to manage data for example. Then it will also bring much more canister installations and much more cycle concept consumption to ICP. Yeah. So cafu will also bring uh a new wave of uh builders that will join uh the community. So now people instead of actually needing to be developers, they just just need to have ideas and they can talk to AI and generate and deploy those. And one other thing we are bringing also so caffeine we need new features such as being able to upload potentially terabytes of data. uh and at the moment this was are happening directly to ICP but now we've been developing blob storage uh solution also for that and Ian and will talk you and Dave more about this and this is something for example that caffin brings also to the entire ICT community soon yeah everybody that wants to try caffin they can already talk with the master chat just go to caffini and just keep an eye on on x on caffini to see more news thanks.
Thanks a lot for the update Kepler and really looking forward for the next phases and uh so this concludes the update part of the agenda and we can now move to features. We only have one feature today which is about uh chaining system CNS and really an update from Barto on this.
>> Hello everybody, my name is Barto. I'm a software engineer at Definity and I want to share the screen because also the next presentation after this one is a demo for which I wouldn't need screen anyway. So that's why uh starting already now right. So Andre already mentioned what what is a CNS? It's a chain system which which is like a DNS alternative hosted on the IC. Uh it has similar structure but it uh the pack that is being or will be controlled by the NNS will allow new uh functionalities. If you want to know more background please go to the GitHub repo both pointed to here. Uh and also what is the minimal CNS? It's a subset of the CNS uh because CNS in itself is quite complex long-term project and it requires lots of resources to implement and so we wanted to uh see whether well is community interested in such a system is it the right type of system that we should work on. So we trimmed down the design I implemented and uh launched it now and we want to gather metrics and feedback from the community and see uh which direction should we go. uh to just briefly I will not describe the entire structure of the CNS or the minimal CNS but just to mention that the minimal deployments consists of the root of the system plus a TLDD operator for a single top level domain ICP which is also the naming for the uh for for the domain and uh we provide also uh CNS resolvers which are for for the developers to to access the CNS JavaScript and Rust libraries Uh so now it's open for testing. It's prepopulated with domains for well-known canisters and names of all the IC subnets. Uh it supports also reverse lookups uh for well-known kind subnets and anybody can also use it to register test domains. So like testic uh it's without any ownership checks because that's the complex part. Uh but also no stability. So if it gets abused, we might remove some of the uh uh test domains that get registered. Uh again, please go to the uh URL to get details about this. This is a team effort, not only from the but also from the uh community. So I'd like to express also special thanks to Byron Becker from Cyclops who provided both ideas and and code to to to the system.
So this concludes the part on the CNS and I'll steal Andra's role and move on to demos directly and for the demos I'll tell you about uh uh was component model for motoc with motocoo. So what is W was or web assembly component model is an architecture for building uh libraries in Wom and or applications that can uh uh cooperate to uh uh fulfill its uh tasks and the components in the common model are specified with a special language which is kind of like language called wheat and uh specifies what uh functionality is being imported or exported uh by the component and one can combine various components into bigger application using spec something called web specification I'm mentioning it because that's going to be important in in the uh what's follows uh now if you are a frequent guest to the global R&Ds about two months ago uh you've seen uh this slide uh where Andre actually described Well, the component in a more uh broader scope and how it fits the IC. But he also showed the this how uh we can compose uh uh software written in different languages to run on the IC. Well, this is the plan. I'm actually this is already a pretty complex uh task and actually I mentioned we to the the interface. So I'm adding here also this uh parts here now. And we're going to talk about this.
Yeah. So, so how do we actually uh enable in Motoco access to the components? Well, and why do we want to actually enable it? Well, some of the reasons Andrew mentioned two months ago, but let me maybe repeat. So, Moto needs really more libraries. It's a new language, so it doesn't have the kind of long history and lots of uh libraries that could be used to write your project. On the other hand, lots of libraries exist in other languages in the old some older languages which provide the functionality but Motoco cannot access them. So, so we would like to uh change that and make it available to Motoco developers as well. So, please uh review the global rend from two months ago if you want to know more about the motivation.
So, how do we go about this? Well, we modifi mod modify the motor com compiler to enable this cooperation and provide tooling and to make this happen for the IC, we would have to also complete what was announced uh two months ago now. So, how do we want to make it? I mean, we've seen on the image that it's like there are many parts of the system and then weak whack and and and other things that you would have to use to to make it work. Well, that that might be overwhelming especially if you are also new to Motoco. So you uh what we really want is to make it easy so that if you want to use any component model uh any component functionality you proceed as you are uh working so far with Moto packages. So you can import something from from somewhere and then uh use it as as usual and with all the type checking as as you are used to and also you don't have to learn about components with uh whack and and other things. The only thing that we cannot avoid is that well the resulting binary will be a components different format that what we are producing right now. Now right now we are producing wasn't uh uh uh modules not components. So that's why we decided that that we would like to maybe show it to the developer you are actually using components but that's the only thing I mean we could use here the say n o or some some other prefix we decide for component so that people are aware that the binary that it produc is different.
So now going back to the image that you've seen. Well, it is complex but we try to make take this complexity away. So we want to will encapsulate everything that's happening here as everything will happen automatically. You will not have to generate any we or files and also when you import the functionality you will import it through a motoc API. So this is what our current prototype achieves. So no wasn't tool experts needed. It has a limited scope. It has only it works for simple functions without the state or and supports basic types and it's only for wasn't 32. That's a current uh restriction of of components themselves. Right? So so they do not support wasn't 64 yet. But this is also a work in progress. another uh thing that we have to do still manually is uh motocoo bindings. So this is this part uh from uh wit file but this is very simple and we will provide also like in as next steps we will provide them yeah uh with ben in an automated way and of course yeah so we want to make the code available behind the flag so that you can also experiment with this and when the f fe future is uh ready uh for components we also want to support 64.
Now, this is a multi- team effort. Uh, but again, I'd like to express special thanks to Andre, Kristoff, Cladio, Camil, and Ryan. And now, let's move to the demo. Uh, hope now you can see some code here. So, in the demo, we have two components. One is just some dummy one meet and greet, which exports functionalities for greeting. say hello say bye and well we have the bike has two has a formal and informal version it's just to use arguments of different different types it has also some other functionalities here but these are just for testing so I will not talk about that let's take a look at how the wheat file looks for that just so that you get a list of how things look like in this world so but these are the important functions that we have and and this is how they are being wrapped into wasn't into Motoco. The other one, the other component we have is is more useful hopefully. So it's actually it's using existing trust libraries uh for verifying BLS or kind of signing measures and it kind of provides some either direct the same interface that that these functions or here we have just also again to test wraps them into some structure and then tries to do some type checking and so on. Right? And again so we have the wheat file with these three functions and and they're wrapping into the motoc.
Now how do we use this in the program? As I said we want to have really imports in the same way as we do it uh right now in motocoo. So there are different variants. You can also do the renaming or you can just import directly from a file. And then uh for example then like for the meet and greet you can you we have these two functionalities say hello and say bye right with different arguments. So uh this is what we can call now for the other for BLS signatures we just so here we use the the renamed import right uh and so we are calling it here and also checking the results if and also do some test reporting for this uh version where we have the password this comes here. So if you want to do this with packaged arguments. So we are passing here some dummy arguments which are not not properly packed here are packaged but still the values are dummy. So that should fail as well right. Okay.
So how do we work with this uh so let me compile this. So what happens now is so what what the compiler does it analyzes the program and it creates all this whe file on the fly it drives them into temporary files and then later embeds like creates a motoc component and composes the component and the final binary that you can run is written to a file and then now you can run it as a time so we cannot run it on the IC yet but can run it on And you can see all the things that were happening. So that indeed kind of the functionality that we are testing works as expected and also the error reporting works. And uh yeah I think I'm almost on time but let me maybe mention one more thing. Uh I I said about the uh test here I said that we have this uh type checking. So if I say I change it here and want to uh give a different like wrong type of argument then of course the compiler will complain. So, so now I can change it back and then the compiler will work again. Okay. Uh that's everything I wanted to show you. Uh thank you very much for attention and stopping sharing and giving back to you.
Thank you. >> Thanks both for both presentation. Uh we can now move to the next uh update and demo which is about kakin storage that both uh kebab and sana mentioned earlier in the talk and uh Ian please.
>> Thanks Andrea. My name is Ivonan. I'm director of research here at Definity and I hope there's now enough hype uh with Sam that Sam and Kepler built up. So now what is this cafe in blob storage? But before we go there, why do we have it? Um, if you wonder what the most popular um apps on the internet are, here's the list. And what do they all have in common? They all store tons of data, photos, videos, etc. And now that Caffeine makes it easy to build such applications, we need a way to deal with this data on the internet computer. Imagine a sovereign file sharing app. Um, if you there if you have like one TB of user files and you want to store that in KA memory, that's like uh $5,000 a year. Not exactly cheap.
And if we go to the next slide, um, we have the the we can benefit from the nice property that actually for many applications the majority of data is written once and then read many times. And for this type of data, the ability to change individual bytes is unnecessary. What is crucial though is that integrity is preserved and that um the cost and speed um are yeah fit well with with the requirements of the app and that's why we have been working on a dedicated storage service for apps generated by caffeine. We launched it last Friday and it has been um used to create quite a few apps successfully since. For this service, we had to write code for the front ends and backends that caffeine generates and teach Caffeine's AI how to use it. And we also built um two new components. On the one hand, the cashier canister which is responsible for um billing in cycles and a new offchain service um called the storage agent that is used to upload data to a storage system if authorization is there and notify the cashier canister of the resources that have been used um to bill for them um appropriately and also for download.
Let's look at this in a bit more detail on the next slide. The core business logic of your application um and the front end code generated both by caffeine. They still reside within canisters as always. And now let's look at what happens when a user interacts with the front end for example on their phone and wants to upload a large file. Then on the next slide you see that um the application will actually not store the file itself in the canister. Instead the front end computes a cryptographic hash of the files data and stores that onchain and this then acts as secure verifiable reference which can be used to detect tampering and when a file is no longer needed this hash can be dropped from the backend canister. the actual heavy data files, images, videos, documents are then sent to the storage agent.
And on the next slide you see that before um accepting an upload, the storage agent checks for authorization by the backend canister integrity and uh whether the backend canister has a high enough balance at the cashier. And if all this works out then it actually stores the data um on a dedicated storage system. And um on the next slide you see how the download works. It's um very very similar to the flow I've just described. And um after front end downloaded a file from the storage agent, it can check if the hash matches um and verify that the data still is the same.
On the next slide, we will see how paying for the storage service is done in cycles and this works as follows. Periodically, the agent asks the backend canister for the hashes of the data that is no longer needed and then removes the corresponding data from the storage system. This relies on tracking weak references being garbage collected and um this is a very nice piece of work that was recently added to Motocore. Then after the deletion part has um been done, it sends a summary of the resource usage um the amount of data stored, the number of upload and download requests etc to the cashier canister and then the cashier adjusts the backend canisters balance accordingly. If the canister's balance at the cashier reaches zero, the storage agent um ceases to serve new uploads and download requests, but it will retain the data for a few weeks. When the cashier observes that the backend canister balance is low, it will notify the backend canister um to make it send a refill of cycles and then the service is automatically reinstated.
In summary, Caffine builds canisters that use this storage service out of the box. It creates the backend canisters that automatically provide the necessary endpoints and to front ends to register the data and to the agent for the deletion and not notification for the cashier and also the front ends that are being built by caffeine. They are equipped with the libraries that handle data registration, upload, download, etc. And um on the next slide um there's the good news that Kepler alluded to earlier already that this is actually just the first step. Right now we have a very basic version of this service running and we are working hard to make it more robust to support the general availability um of caffeine and then at the later stage we will decentralize this service with more agents and a very cool new erasia coding protocol that beats existing solutions and allows us to have very low replication factor for the data we store. And last but not least, we will integrate this service into the internet computer protocol so all canisters can benefit from it. And uh now I hope you want to see this in action. Um Dave will take over. And while he gets ready, I would like to take the opportunity and to thank everyone involved. This was really a cross team effort. Um thanks um for everyone who helped us make this a reality.
All right. So, um, thank you for the technical details, you want an I'm going to take you on a quick tour on how this looks with caffeine in production. We're not going to go through the prompting step because that's already been done. That would be outside of the scope of this demo, but I'm just going to show you I'm going to take you through the app that I created to showcase this feature. So I created this like online learning course platform where obviously the goal is you can share videos, you can share PDFs, large files with everyone who wants to learn, wants to enjoy those courses and uh you can already see here like the media integrated embedded. But first what I want to show or want to start with I want to create a course. So I'm going to have like an amazing course. So, I'm going to select a couple of videos and PDFs that I want to add to this course and then I can upload those files. And what we can see here is basically we have the front end upload those four files in parallel um to the blob storage to the immutable blob storage. And you can see here we have like a small PDF 1.3 megabytes. We have a 5 megabyte PDF. We have a 100 megabyte file and a 37 mgabyte video. So those two are videos and um yeah, while they're uploading I can tell you that we already identified a couple of key points that we uh how we can make this faster and we can and we will make this faster. As you heard we're just getting started on all this um blob storage and maturity. But you can see that we can already work um yeah seamlessly with with any kind of data, not just PDF or MP4s. That's just what I've chosen to showcase here. But um yeah, all of those files have been successfully uploaded. We can add some description, which I'm not going to do here. But then we can create the course and we see it's done here. We can like set it to public. And now you should be able to see the course that we just uploaded our amazing course here.
>> In this video, we take a look at the >> let's not uh play this right away, but you can see here we can watch the video, we can skim through it, we can look at the PDF and we can also jump to the other chapter and yeah, again like just look at all the files we have up we we've just uploaded. Obviously these are kind of small files you could also already upload to your canister storage but then we can also look at larger >> ideas >> larger uh videos uh for example this is a 2.5 hour video and again here we can like just seamlessly scroll through the the timeline and look at it and uh last but not least I want to show you um um probably well-known video in the public domain which is Big Buck Bunny and you can see in this case that we and stream like this. I think it's a 4K video um yeah without any problem from the blob storage to your caffeine generated app whatever it might be might be something like an online uh learning course platform or a YouTube clone or whatever your own favorite videos from your vacation. And last but not least, I can also show you that this is not just an embedded YouTube video. If you look here into the resources tab, we can see that when I jump to a new time point, we can see that it downloads those blobs in real time. And if you look at the actual request URL, we can see that we're accessing the blob caffeine.ai URL which is serving those blobs. Um yeah, I think that's all I have from the demo here. And yeah, back to you.
>> Thanks. That was really great to see in action. And uh we are already at the last demo of the day and uh we had Dominic from Diode who's uh going to tell us a bit more about diode and also about how they integrated with best keys. Do you want to share the screen Dominic?
>> Yeah, I can just share my screen here and then we can go through that. So let let's start with dire. So I'm actually super excited about what I just saw here from Ivonne and al also Bush because I think we're going to need both of that. um as as we go ahead here. Um so to give some context, diode is like you can imagine like the Microsoft team slack replacement that is living on chain and is using the chain to provide self-custody for our customers. And the reason why exists is really because of this growing tension that we see in the uh international legal system between like people owning the data like the people producing the data. you might be a European citizen and then the company actually operating like Slack as an a US company under different laws and this US laws, EU laws but also Brezil for example, also Russia, also China, they have all this problem that if you bring these two different entities into the mix, you create problems like who actually owns the data? But what did the governing law on this data? And so far before diode like the answer was always to like you could self-host and like run your own servers and do all of that or you go like all cloud and you try to deal with the trouble like some of these uh legal troubles are still ongoing like Microsoft is still fighting in Ireland. There's like a big GDPR clause and we don't know what's going to happen with like the cloud act. So what we want to do is we want to say like let's try to shift this problem technically right there's for sure regulation making progress but we can also solve it technically and the prime example of like something that solves uh an issue like this is has been email like you don't see the same problem that you see with Slack and WhatsApp and all of these tools was email because email is a pure protocol and email is amazing because you can choose out of a 100 different email clients and these email clients do not own your data right the data and the email protocol is independent of that. And so this is where we came in and saw actually with blockchain now you can prov provide and create all these experiences that before blockchain were not possible because you have state on the chain. You can create actually a Slack clone. You can create actually a Google Drive clone. You can actually create all of these tools while keeping full self custody like you own the data. You have control of the data. It's just a protocol.
So I have a screenshot here of the diode app. I want to try to be conscious of the time. So let's get into this a little bit. Um so it's not we don't focus on like messaging c consumer to consumer and like messenger like WhatsApp and signal. This is really focused on you have a company, you have an organization. You create an organization and you tell your employees, your members, your um that you want to use this tool to collaborate, exchange your chat messages, exchange your files and so on. And that has been from the ground up being a peer-to-peer pro protocol. So that means here these yellow boxes are diode clients being online at the same time. And so if I send a chat message, if I send a file, those are being sent from one client to the next client as they're being online. And this works great as long as everyone is online and the things just start happening. But what we see more and more is that actually as people uh become more mobile first users, often what they do is they open the app, they send a file, they assume the file is being sent, and then they close the app. Right? So what's happening is here we have Frankie and Loki. Frankie is coming online trying to send the file but actually nobody from the peer group is online. So this peer-to-peer network doesn't work if nobody's online. And this is where we started to um bring in canisters. So we have this uh we call it the zone availability canister because when you create a group that's we call this in in diode lingo a zone. And so you create this canista that is always online even if nobody else is online in your from your peer-to-peer group. The canista can be that bridge to store data to store messages to be the one to ensure that once Loki comes online later when Frankie is offline the data is traveling between those. So again by default the diet app is like a local first self- custody app but you can use like a canista to ensure like this mobile experience or like when people are offline experiences is working really really well and that's what we have started doing here. I want to talk a bit about what we did here most recently and that I mean that just comes in back to what Ivonne said like we we did what everyone else did. We built built like a file system API inside a canista. So we are able to upload actually all of the files that people share. We started with messages, then we added metadata and now finally we have also added all the files that you can drag into your zone. And so there's some change tracking. So you can like if you keep your local system in total sync. You always only want to get like the most recent files because everything else you already have. So there's a change tracking system. But also if you have no files like on mobile, usually people don't sync all the files from a zone. you only want to be able to browse. So there's a directory index and you can just browse by directories as well. And all of that we keep in this new Motoco feature. So we we are big Motoco users and Bartraush I'm waiting for that was a module to be able to use all of the cool Rust modules that I've seen. There unfortunately not so many Motoco packages always. So we're using the new enhanced article persistence for most of the like the directory tree, all the meta data and everything that points and describes the files. But then the blob storage itself like the big bigger files are in a stable region right that can grow but as we as we just heard they're also limited they're also pretty expensive and and so we keep them in the stable region in a ring buffer so as a zone owner you can decide I want to allocate like a gigabyte of data maybe 5 GB of data and this is how much I want to expense for this kind of caching in the availability canister so everyone else can get the files but when when it rolls over basically some files are getting deleted. I'm really looking forward to play with a new blob storage in the caffine system. So I hope that comes uh is going to be available to us very soon. I want to jump in a bit deeper because I'm running out of time here into encryption and that keys. So everything we store on in the canister is being encrypted with a scheme that is called the bit message scheme. It's pretty
old. It was when blockchain came up and Bitcoin came up. It's like a system established for really messaging in a state where it's like, um, non-interactive. So you put the message somewhere and someone else picks up the message later on. It's a pretty simple system.
It's using alometric keys, which is really nice here because it means you only need a public key that you can make available to anyone to read. And then using that public key, you can encrypt a ton of files and just keep uploading. There's nothing you need to do to basically upload and send files. But if you want to be able to download and read the files, you need the private key and you need to get access to that private key.
Put some links here into the implementation, like the Bitmessage documentation here, but also our implementation of that in Alex here. And so this kind of brings the same problem with it. Now we have this private key that you need to read the data. But again, like, what if Loki is offline now? So Frankie wants to send the file. We solved the problem with the file. It's in the blob storage. What do we do with the private key? How can we send the private key without exposing it to like everyone in the world or to everyone who can read the canisters, to the node operators in the worst case? How can we protect that?
And so there's this technology called, uh, the Vet Keys. And this is something that we, uh, started using and are using in production right now to protect these, uh, these keys. Um, I'm going to jump a little bit into, um, how my understanding is as we implemented this, what, how the Vet Key is actually protecting it. But, um, to make it really short here, we take the public key, we generate basically the key pair, uh, off-chain, private key, public key. So that allows a peer-to-peer system to exchange it if people are online actually. But if people are not online, it, uh, uploads a protected version of the key into the canister. And that protected version of the key is protected using this derivation function, um, of the Vet Keys. And when we want to restore that, in the same way, Loki here, he can get the protected part, derive using, uh, the Vet Key mechanism, the missing piece of it, and restore the private key.
Now, this is like the pseudo-curve math, um, that you can imagine here. So what's happening is when Loki is wanting to restore the key, he's first sending like a transport key. Again, generates, uh, a new temporal key, a public key, and then sends that request to the first, uh, to the node system of the canisters. And they basically multiply or add, depending on the terminology, to this transport key their shard, their fractional knowledge about the secret here, from one to the next to the next to the next. And so that means they, they add this to the transport public key. They, each node itself doesn't know the full private answer. And even the results out of all of this additions is still encrypted in a way that none of the nodes can read it. They don't know what actually is behind the secret.
And then when they send it downstream back to Loki, and he got it, he can then use a private part of this transport key that he just generated before that to decrypt it and get the real key part that he then uses to restore, um, the Bitmessage private key, which he can then use to read and restore the data. We do a very simple validation that's kind of built in in the Bitmessage system because it's a public-private key part and the public key is public. We actually make it public knowledge. Um, once you get the private key from the canisters restored, he can check that this private key does in fact return the correct public key.
So that's, um, diving into Vet Keys here. I'm not going to explain you guys what a canister is. There's more if you want to know about this, uh, on our website on Kagram. You can reach us and of course, we also have a direct group. But I also want to really quickly show the app. So this is, uh, this is the app in action. Um, we have, uh, a couple of team members here. We have some storage. Uh, I can see in the zone identification, there's a canister connected to that. That's actually a production canister. Shows a version number, some of the total storage information from stable memory. And what I can do now is, these are all self-custody friends. Um, I can create a new folder and I can choose a file. Yeah, let's do this. So we're programming everything we do in Alexia is like a pretty French programming language. But, um, so we also created an ICP agent in Alex here. So this is being uploaded now to the canister. You can see there's a progress bar. This is green because actually nobody else is online here. So Rosie, Herbert, Loki, they're being gray indicators. They're not online. And because they're not online, it cannot be synced. But it can still be uploaded to the canister. I mean, this is very similar to the demo we just earlier saw. But this is actually in the canister storage. As soon as we get our hands on large blob storage, we'll store it there.
So we want to quit Frankie. I'm going to from the command line run Loki's account here. And so this is the case we had in the picture where now people are uploading downloading asynchronously while the other one is offline. So the peer-to-peer system cannot do it alone. And so Loki, by when he comes online, he doesn't see the directory. He's not aware of this. But in the background, the canister is being checked and eventually the directory pops up. The file is seen, it's from the directory index, seen that this file exists, but it's not yet synced. So we can request it to be synced locally because locally, uh, locally wants to work with the picture. And we can see there is an August summer event in, uh, in Berlin for Alexia engineers. And that's it, really end-to-end. So this is the way we bring this down to, uh, Loki. It's encrypted using Vet Keys and there's a key recovery using the newest canister features. If you want to see the code, I linked that in the presentation. It's all open source how the canister is using it. And I'm happy to be there for any questions. Thank you so much.
>> Thanks Alain. This is really exciting to see all these things actually coming together and, uh, CDC used as infrastructure for your, uh, service. Uh, we are at the end of this month's global R&D and, uh, thanks a lot for everybody who attended and see you next time.
[Music]