Data + AI Summit Keynote Thursday - YouTube

Tags

Top Highlights

• databricks connect has been upgraded to build on spark connect

• Apache spark has become the de facto big data processing framework

• a culture of incorporating bleeding edge research ideas

• Spar has been the most actively developed project in Big Data with over 3600 contributors and forced out forty thousand commits is remarkable for a 10-year history

• a few principles behind the project really worth highlighting the first is that the focus on simple expressive and modular apis with well-defined semantics it makes the program easier to write but also more importantly this API sufficiently abstracted so they allow the backhand to optimize over time without the programmers having to actually change the code they write the second which is often overlooked is spark can run virtually everywhere you 35:14 can start developing your spark programs on your laptop without any internet connectivity I was literally coding the last time I was on the airplane spark programs running on my laptop and you can use your cicd tools in your own network environment and you can publish your spark program to the private cloud or the public Cloud for execution this is very important because you don't have to depend on any third party everything is self-contained you don't have to handle any third party just to even test your program

• now last but not least the multi-paradigm extensibility of the Project Spark is available there's many facets to that Sparks available in virtually so all the most important popular programming languages and data and AI started with Scala we added Java R python NCC coordinator but there are other ways to get a stand Spark for example data sources and Federation it's been extended to virtually all data sources any data source you name you could probably find the open source implementation of that data source for spark out there

• spark is a very important part of data science and AI the four most popular AI Frameworks right now are probably launching hugging phase Pi torsion XG boost and all four have native spark Integrations

• Project's not sitting idle right we're introducing the community is working a massive number of improvements literally thousands of them for Every Spark release we won't be able to go through all of them but I want to highlight three today that's my personal

• spark connect is GA in spark 3. 37:12 4 now it creates a narrow waste for Apache spark computation and this narrow waste can be leveraged to create thin clients that can be embedded into different programming languages applications and Edge devices so what are the use cases for Smart Connect first you can use spark connect to connect to a remote spark deployment in the cloud in your private

• in spark 334 the latest release you can see full signature along with the full documentation in autocomplete directly making it much easier to write your spark programs

• creating a new library that introduced native research for data and a lot of other capabilities make it substantially easier to test your Pi spark program with all of this work we're really making python a first class citizen going all the way from the end users experience to the experience of framework developers and more sophisticated users that can actually extend spark itself

• excited to introduce what we call the English SDA SDK for spark it's a new open source project they help you author spark code with prom engineering already done by the spark experts to minimize anti-patterns

• English SDA for Apache spark does not just help you write your basic spark code it actually helps you with all the 47:12 stages of your data science or data engineering program going from ingestion transformation verification explanation to applauding all right go check it out at Pi spark.ai for the English SDK for Apache Spark now

• elta is fast it processes in some ingestion use cases more than 40 million events per second and it's popular we had over 500 million downloads in the last year one of the reasons it's so popular is because it's reliable it's been battle 48:58 tested by more than 10 000 companies in production it's open with over 500 contributors adding into the ecosystem and we're not slowing down we've got over 80 features that we added in the last year and I'm going to talk about some of them today one of the really cool things about Delta is it's the fastest lake house format

• we're faster not only at loading data but then also querying the data once it's been loaded into a table one

• Delta sharing it is the only open protocol for sharing massive amounts of data without being locked into a single vendor's compute platform the way it works is we actually sign individual parquet files on demand so you're still securely sharing your data but you're always sharing the latest copy and you don't have to make 50:33 extra stale copies of it

• new features that we've added including support for structured streaming as you can imagine when you have a huge data set it just doesn't make sense to scan the whole data set over and over and over again you want an efficient protocol that 51:32 tells you only what's changed since the last time you read it and so now it is deeply integrated with structured streaming we've done tons of improvements on the back end and so now we've improved query latency by up to 50x we have full support for oauth 2 spark and pandas and then finally for the CFOs in the audience there's a pretty exciting development when you shared Delta sharing data on cloudflare you can now do that with zero egress fees

• Delta 3.0 we have a whole slew of new features and I want to focus on three of my favorite the Delta kernel uniform and liquid clustering

• Delta did multi-dimensional data clustering liquid partitioning is up to 2. 53:59 5 times faster

• now as much work as we do on the 55:29 ecosystem there will always be other engines and we actually think that's great it turns out that any lake house format is better than a proprietary system but since we can't control which formats these other engines are going to be able to query this is a big problem for some of you deciders in the audience you don't want to pick the wrong format today and then find out tomorrow that it is forcing you to use some proprietary system that doesn't support Delta well you have to convert all of your data at 55:58 that point that's a very scary Prospect but if you squint a little bit it turns out that's actually probably not necessary if you look under the covers all three lake house formats are based on the same fundamental principle which is an age-old technique in databases multiversion concurrency control basically all the different systems store parquet files when they make changes rather than update the parquet file in place they make a copy of that file and then the real transactional magic comes from this extra metadata 56:29 that sits on top and says for the this current version of the table which parquet file should I read and since all systems are based on the same underlying principle we're really excited to announce Delta uniform what Delta uniform does is we have one single set of parquet files and then we can create the metadata of any lake house format so you can query your Delta table anywhere in the open lake house ecosystem so we're going to take this fragmented ecosystem and bring it all into Delta and you might ask yourself is it fast it 57:12 sounds really expensive to create all of these different you know copies of the metadata and it turns out as I said before the metadata is actually a pretty small part of it it actually costs less than five percent on your right times to enable extra formats and what's even more exciting is it turns out Delta is better at writing Iceberg tables than iceberg is when you use Delta to produce an iceberg table yeah that's like yeah because of the advanced clustering I was just talking about when you use Delta to 57:42 produce that Iceberg table you have better parquet files and thus your queries run faster so this all sounds a little bit too good to be true and it wouldn't be a spark Summit if we didn't do a demo so I'm going to move over to my laptop and let's see how this works it's a big stage foreign okay so here we are in a databricks notebook and you can see a very standard create Delta table statement and all I'm adding to it is this one extra table Property Delta Universal format enabled formats and I can just list which other 58:22 formats we want we're releasing with full support for Iceberg and we're working with the community to add support for hoodie so I'm going to go ahead and run this command and then I'm going to switch over to another cloud data warehouse and as you can see we've got our table here and if we look at it Google bigquery thinks this is an iceberg table so let's go ahead and create a new query on it go ahead and add a star here and I will click run and there it is it's actually a Delta table 58:58 pretty cool [Applause] I wanted to show at least one other cloud data warehouse but unfortunately their open format support is still in private preview and I couldn't get access but if you do have access I encourage you to check this out and with that if you'd like to learn more there's a bunch of other sessions today you can learn all about the exactly how uniform is implemented in this session on Iceberg and Hoodie in Delta Lake you can hear one of my favorite use cases where they Adobe actually converted from 59:36 uh Iceberg to Delta and you can also learn how to build your own Delta connectors with the Delta kernel thank you so much [Applause] please welcome to the stage original creator of Apache spark and ml flow Chief technologist matte zaharia [Music] okay hey everyone um very excited to be here to host our next event which is an AI faculty panel with faculty from some of the top AI groups in the world so we've heard a lot from folks in business uh from customers about AI but I also want us to hear a little bit about what's happening in AI 1:00:29 researcher and you know what these researchers who are leading the way are excited about next so I'm really excited to introduce three members of the panel today we have Mike carbon who is a professor at MIT and also a co-founder of Mosaic ml when we started the panel you know it was before anything was even happening with Mosaic ML and databricks so awesome to have them here we also have Don song from UC Berkeley who is an expert in AI as well as security and Daniel larus who is the director of MIT C cell you know arguably the birthplace 1:01:01 of AI research in general so welcome them all to the stage with me [Applause] thank you and huge huge pops for Daniela for coming here despite Carter's being injured she's she's amazing so you know we didn't even know about it until until very recently so uh yeah um so so welcome everyone so let's start with uh with some introductions um if you can just each tell us you know kind of what you work on what what have you been doing you know recently that you're excited about in AI research to 1:01:50 give the audience a sense of it and maybe Mike you can begin oh yeah sure so I'm here I guess I'm here with my two hats on so my academic hat where I've been doing research for about 15 years and of course my Mosaic ml hat where I'm a founding advisor and you know having a great time doing work there as well and of course you know potentially our plans to join the databricks family shortly and so if you look back to the history of what I've done where I come from I come from computer systems and 1:02:18 particularly my entire interest has always been in I don't make computers go fast how do I make them more efficient and I do a lot of work as well in the programming languages and compilation space so how do we look at the structure and organization of programs again to make them faster and more efficient press for this audience out here in the data space and the database space thinking about things like query optimizers that's where I spent a lot of my time but you know how did I get into AI well and some of the work that I've 1:02:45 done with my students over the past several years what we've done is we looked at the structure and organizations of neural networks themselves I notify there's parts that perhaps don't matter so we can take make these very large models turn them into much smaller models perhaps an order of magnitude smaller but still train just as well and still give you good amazing results and over at mosaic ml these perspectives towards efficiency is what we've been driving based on those insights of how can we take these large 1:03:12 models and make them much more accessible for for you and everyone else that's out there right these models aren't only in the hands of the largest organizations they can be in everyone's hands great yeah Don great yes um so I'm a professor increase Berkeley and I also have a startup working at the intersection of AI security and decentralization as well so um there are so many exciting you know areas that we have been working in the areas related to AI so here I'll just talk briefly about the areas at the 1:03:47 intersection of AI and security so given all the amazing advancements of AI capabilities LMS and so on so of course one application domain that's really exciting is in the security domain all these advancements in LMS and so on can really help a lot in the Security application domain for example some of our recent work show that by even just using you know GPT and these large language models actually it can really help significantly improve for example security adults of application for example we were able to show that it can 1:04:27 significantly improve the manual audit efforts for security for smart contracts and also we show that by using these large language models change um for example blockchain transactions and their execution choices we can identify anomalous and malicious transactions and for a broad spectrum of different applications that we examines these trained models can actually Rank and attack transactions in the real world to be actually the most anomalous transactions for over a dozens of different real world applications and 1:05:09 these attack transactions actually have caused close to 300 million dollars worth of damage in the real world so this is just another example illustrating the power of these new large language models and the AI capabilities and on the other hand of course these type of capabilities also pose a lot of challenges that's also something that we have been studying for example showing that these large language models they have big privacy leakage problems and also some of our recent work in collaboration with the 1:05:41 meta has been on how we can enable privacy preserving AI model fairness assessments and this actually is the first half is Kind large-scale real world rollout that's been loaded to Instagram users to help in for the first time in a privacy preserving way to assess the fairness of AI models that's actually being used in the real world so these are some examples of exciting areas that we've been working in very cool and Daniela well so I'm also here with multiple hats I'm a professor at MIT I I run csail my students my former 1:06:17 students and I have a number of companies in the space of AI and it's just such an extraordinary moment for for our field and in particular for AI now at MIT what I'm most interested in is is really asking some foundational questions about what AI can achieve and so if we think about the fact that the majority of the successes we hear about today are due to decades-old ideas that are now empowered by data and computation it's kind of natural to ask what else is there because if we do not come up with new ideas in time everyone 1:06:57 will be doing the same thing and the contributions will be increasingly incremental and so I'm super excited about a new approach to machine learning which we call Liquid networks it's a compact approach to machine learning for it's a kind of a continuous time approach to up for applications that have time series data liquid networks are compact provably causal and they have very nice generalization properties when trained in one environment and applied in a different environment with really a huge 1:07:39 distribution shifts and so um yeah so what the world is trying to make networks bigger and bigger I want to make them smaller and so to give you a sense of what you can do with liquid networks um in our work we do a lot of um Edge device and robot applications and so for instance if you want to get a robot car to stay in lane and steer it takes about a hundred thousand diploma networks to get good behavior and it only takes about 19 of our liquid networks and so if you have a solution that involves only on the order of a 1:08:17 couple of uh of of nodes or maybe on the order of tens of nodes you can then extract decision trees to explain how that system makes decisions and in doing so you can begin to address safety critical applications where knowing how the model reaches decisions is very important that's super cool yeah I do think now that we've gotten you know neural networks with gradient descent working people will discover so many other ways of doing you know this kind of computing and in a much more controllable way so 1:08:48 super exciting to hear about that um so one question I I had for all of you is um you know as researchers you see a lot of the emerging ideas and Ai and here we have an audience with many practitioners who are you know building things every day so what's an idea or an approach you think is is coming up in UI that um you know in an AI that is looking promising and that you think practitioners need to know more about you know that they aren't paying enough attention to um so curious what each of you think 1:09:18 about uh about that if whoever wants to yeah the eyes around me all right uh let's see so I guess so we're at a big data conference I guess I actually want to say small data uh and so what I mean by that is there have been some interesting results um some things that we've been working on some things that other people have been working on uh and a highlight I think you can look at from the past couple weeks so maybe a month old at this point there's this paper textbooks are all you need now it's important if 1:09:47 you see a paper that's out there in Academia sometimes it's less about the result and more about the idea as we're talking about here but what they were trying to do they they wanted to go out and build a code model like a code generation model sort of like what you're seeing yesterday in the lake house IQ example the lake house Q demo where large language model is going to generate code and the typical way that you're going to do that now is you can find as much data as you possibly can scrape the internet 1:10:11 stack Overflow all these data sets that are large and pretty dirty in some ways throw them into a model and this thing will amazingly generate code really great but these models are large right we're looking at chat GPT you know in that at least in the 3.5 chat gbt with gbt 3.5 and that 150 175 billion parameter range these are large these are massive right and we we think these models are perhaps out of the reach of many but in this paper what they did is they find again small data if you do a very good job of trying to curate your 1:10:43 data you can find these opportunities to process on less data orders of magnitude less data and with orders of magnitude smaller models so they had a 1.3 billion parameter model that was besting some of the best models that are out there in the 15 billion parameter range and even exceeding at least in their domain evaluation the results that you're getting out of chat gbt driven by GPT 3. 1:11:08 5 and so I think that's the future the future of Focus right Chad gbt is absolutely amazing you know it knows everything in all these different domains but when you can focus on your particular domain in your use case I think they're really good opportunities that for building these models and building these models yourself at a scale is accessible to you and so that's the direction I see because I know everybody's thinking about chat GPT but these new coming directions with these smaller models I think are very promising very cool yeah 1:11:36 um I think uh for practitioners in general you know practitioners are really excited about these new capabilities I want to use them in different applications and so on one hand it's really exciting to see all the great progress and what you can cool things you can do about it but on the other hand I think one thing that usually practitioners maybe don't think enough about that's the thing that I wanted to emphasize here is that often because these services are exposed to you know the general users and so on 1:12:09 like you have to think about how these Services may be misused for example um these air models with our work and with you know other research work in the community these models have a lot of security privacy challenges and issues and they can be easily you know this kind of jailbreak and also they can be Foods in giving wrong answers and also as practitioners use um models out there actually there has been you know our work and others work have shown you can very easily implant for example this we call them back doors 1:12:45 or children's into these models that's very very stealthy you don't even know that they're there but actually um then the models can be triggered under specific conditions to misbehave so this is just the tip of the iceberg there are lots and lots of these different types of security privacy and other types of transporting issues with these air models and how they can be properly used in the application domain in a responsible way and I think one of the really big challenges actually for practitioners for deploying these models 1:13:21 is how do you know when the models are actually ready to be deployed how even to measure how the model essentially behaves and how what are the Matrix for measuring the traffic and the different aspects of the model so one of our recent work uh with my you know Wonderful many collaborators is called decoding trust you can actually go to the website decoding trust.github. 1:13:50 io to look at more details so this is uh the first comprehensive systematic actual evaluation framework that covers a full broad spectrum of different aspects different metrics in the transportation software models and this is only just the first step we hope that this as open source framework we can encourage the community to come together to further develop this to really help practitioners to know better how their models are doing besides just the performance perspective but actually more in the Sprout trustworthiness 1:14:23 perspective to really know when the the model actually is ready to be deployed if not what are the issues and so on very cool yeah maybe so we can talk for a long time about the excitement of of the work in Academia because um despite the fact that we have such extraordinary Potential from AI there are a lot of problems that need to be addressed in May in order to make AI safe and trustworthy and environmentally okay and easy to deploy I guess Don was talking about how do you know when the model is ready thinking about rigorous 1:15:01 approaches to testing and evaluation is super important thinking about how we how we build trust in in how we use these how we use AI is super important but if I can pop up at one level I would say that when we when I think about how AI is used today the basically three categories of AI Solutions we have ai solutions that are primarily about pattern recognition and and this is where deep learning and very and kind of fit full and then there are AI solutions that are primarily about deciding what to do and this is the kind of the body 1:15:40 of work around reinforcement learning and then there is the the group of AI solutions that are all about generating new things and this is the generative AI space which includes tools like GPT and in each of these areas the Academia is actually working to better understand what kind of solutions we get to better to increase the representation in other words what can these uh what can the models represent and recognize and also to to really understand the properties and I will tell you that in each of these three categories we have issues we have 1:16:19 issues around data because they all require a lot of data and um that means that the computation is huge that also means that there is a large environmental footprint I mean did you know that deploying a very small for today's standards model today releases 626 000 pounds of carbon dioxide in the atmosphere this is equivalent to the lifetime emissions of five cars so as we think about deploying AI in the future it's important to keep these numbers in mind because we want tools that support Humanity but also the planet 1:16:57 so we have to deal with data we have to deal with computation we also have to understand the black box and ensure trust and privacy in our Solutions and so the good news that is that the Academia is working on all these problems and next year when we come together at this meeting we can report to you on on more achievements makes sense cool yeah I think we have time for a couple more questions so I uh I'll uh so let me uh try to think uh about some of the more interesting ones so okay so one question um I have for everyone uh that I think 1:17:33 is on everyone's Minds here is you know what do you think about AI becoming sort of democratized or commoditized so like on the one hand things like like computer vision you can run you know pretty much anything that state of the art on your mobile phone on the other hand we've got large language models which are you know extremely large and it seems making them larger makes them even better what do you think about the technology Trends will this be something that you know um uh kind of everyone that that will 1:18:02 get a lot cheaper that everyone can do uh or you know what what are sort of the pro and counter Arguments for that yeah I I've already said it yes I can just say it over and over again yeah I think these are these are the trends that are taking over um and we talk about democratize I I still feel like I want to break that into two different categories uh one is you know we see chat GPS and it's absolutely amazing again like I said you know it knows literature and the Arts Sports pop culture right anything that 1:18:36 you can think of right that I mean it's truly truly revolutionary I have to say when I first sat down and started using it I hadn't expected to see these types of capabilities particularly in the interactive code generation sort of getting back to my domain perhaps in my lifetime truly amazing I think in that space we're already seeing democratization in the sense of increased competition right I mean Bard Claude PI from inflection results that we're putting out at mosaic um there's a there's a hot race right 1:19:05 um yeah the economics are there to bring down the cost of those models I think there's this other direction as well on can you build these things yourself right perhaps it's great that there are these closed models that are out there that you have access to but could you build this yourself and I think the trends are are positive there as well so going back to what I said before you know this textbooks are all you need results if you can focus right again so cheap can do everything but if you're 1:19:34 building a biomedical model do you need your model to give you Sports statistics right or talk about Taylor Swift no right but Which hat gbt you're paying for all of that right you're paying in the terms amount of data small needs to be trained on all this data you're paying in terms of the model size the more data the more recall you need over a larger knowledge base the larger the model needs to be to be able to do that effectively so if you can focus there's opportunity there right and textbooks are all you need there's 1:20:05 there's an example there and some work that we've done at mosaic we actually did this in the biomedical setting where we trained a small by again relative measures three billion parameter model on PubMed which is this huge biomedical Corpus that's out there biomedical papers scientific papers and then fine-tune this to actually respond to questions on the USMLE so the U. 1:20:26 S medical licensing exam and we set state-of-the-art results of course I was back in December and we were able to beat models that were 120 billion parameters at the time because we were able to focus so for me that it's it's about focus and I think those Trends are in the favor everyone sitting in this room where you've got really interesting data and it's just about Focus being on that data and building malls around that space I can jump in uh if you um so I think that there's already democratization 1:20:57 like everyone uses chart GPT no matter what aspect in work and also in life and the tool is extraordinary but um the the concern is that people don't actually understand how it works and so um so this actually causes a lot of problems with um uh with with how the tool is used and how people respond to the fact that now we have this extraordinary tools and um I would say that it's important for the public to really understand what tools we put in their hands so communicating to the public and educating the public 1:21:36 is important and that begins with highlighting that um indeed the tools we have today are democratizing our access to information they're kind of giving a microscope into the the Digital Universe but this microscope is going to make mistakes and so it's not about so so using the tools is not about replacing activities but about augmenting the human with some extraordinary new capabilities I like to tell people that they should think about using an AI tool sort of like they think about an assistant the assistant runs 1:22:14 around looks for patterns brings interesting data for the decision maker to act on the human really still has to be has to stay in charge the other thing I will say that this demo is causing a lot of fear in the population fear that a lot will be replaced by machines and this is just not the case and so we have to be careful with how we how we excite people about being augmented about being empowered and augmented with new tools rather than being replaced by by these tools oh and yeah one uh one more quick fire round because we're we're running 1:22:56 out of time so just you know quick answer uh obviously everyone's very excited about llms now but there are so many other types of data so many other types of applications interactions we can apply AI on uh you know what do you each think is going to be the next day here to get a chat GPT like moment liquid natural Network okay it's very general yeah area area area oh no I'm still processing chat GPS he said short done oh I think I mean also following up a little bit to the pre the previous question and I do think you know this 1:23:35 democratization that there are many different aspects to democratization right the negative results in this space yeah right so like this you can look at uh um it's also open versus close source and uh and also ultimate like we talk about this uh personal assistant that helps helps individuals but who actually controls uh this uh personal assistant whether we want to just have the big pet companies have close-ups models to control that's how we want to have open open source models where actually individuals really can have this kind of 1:24:10 personal assistance that actually enter their control and really try to act uh in their best interests and following uh or the earlier sets is the reason we have some results talking about the challenges for building open versus Source models um we had this paper on the first prime minister of imitating yeah there's not enough yeah right so like it's not so simple you actually really have to do the hard work to improve the performance and of the base model and so on so there's a lot of challenges and how can 1:24:46 the open source Community to come together to make that uh right to really try to close the account yeah so the training process yeah makes sense yeah cool Michael I I think I just want to say multimodal right so yeah I'm required as a programming languages person to say co-generation but um just multimodal so I think we're just in this age where we suddenly have the capability to have multimodal interactions with computers and the fact that that's going to bring so much new capabilities so many people 1:25:20 who could previously only access computation via programming right and perhaps those people were excluded I think that's that's just the future more computation for everyone so I do want to jump in because I only had two words um and I will say that I agree with everything my my wonderful colleagues have said but I think we also need to understand what's happening in the box and so um aiming towards causal models is important yeah and the other thing that's important is to get a better understanding of what's Happening Inside 1:25:50 the Box because it ultimately we will get to the point where we will need certification or we will need some kind of deployment guarantees when we uh when we put AI in service especially of safety critical applications yeah very cool well thanks so much everyone really awesome to have you here yeah foreign [Music] continuing on with our theme of Open Source projects and democratizing data our next speaker is the co-creator of drb I love duckdb it actually has all of the benefits of a real duck when applied to database management including 1:26:53 versatility and resilience but without the Quacks last month alone drb had more than 2 million downloads so please join me in welcoming new ice into the stage [Applause] [Music] thank you Brooke yes hello good morning my name is hannes muleleisen and I'm here today to talk to you about duckdb but I'd like to start with a small story this is my actual car license plate for my first car 20 years ago and what you can probably tell from this is that I may have been a database nerd back then already um I in fact I love these things so much 1:27:44 that I went on and do research and this is where I'm at Mark Grassfield who is here today and we were starting to think about new ways of building database systems and we were doing that in Amsterdam at the CWI which probably you have not heard of but it is the Dutch national research lab for mathematics and computer science and it's the place where python was invented by Guido from Rossum all those years ago so it is had had a certain impact on the world but while we were there we started feeling that there was 1:28:18 something wrong with the world of data and I should explain it's a movie from the 90s in this movie this character tells another character that he knew there was always something wrong with the world but he couldn't quite put his finger on it and we had the same feelings that there was something sort of off with how people were doing stuff with data and that we should probably do something about it and one thing we noticed after you know many years of thinking about that is that what was happening is that we're 1:28:48 people using very big Iron systems like Hadoop to solve fairly small data problems you know stuff that's a bit too big for Excel or pandas or something like that and by using some big iron for something small like here this uh Sledgehammer to Cracker nut apparently that's an expression you just get bad ergonomics in German we have a saying to shoot at sparrows with cannons and in fact there was a development that has sort of supported this where we now have data Lake formats are built on things like Pi 1:29:24 K that actually have the ability of turning a huge data set a very unwieldy thing into something much more manageable and we can with formats like okay we can actually very precisely fetch only data that is relevant to a specific query what also has happened is that we got a disconnect between storage and compute and I know you know this and people have talked about this before but I feel it's the difference between you know knowing that something is happening and you know appreciating all the possible consequences of it and one 1:29:57 of the consequences that these things are disconnected is that we can scale down compute to the appropriate size that's you know sufficient for the problem and you would be really really really surprised how small you can scale down data problems um in fact the laptop in front of you is more powerful than you think so if the data is small enough you know there's no reason not to put it on your laptop another thing that we've noticed while thinking about you know having the Splinter in our minds about Data Systems 1:30:28 was that the community was spending a lot of time thinking about the meat of the Hamburg at a Patty apparently it's called um and this was great you know you can write a lot of interesting research papers about things like join algorithms and things like that but the problem is you're not looking at the whole hamburger right you're ignoring the end-to-end user experience and that was such a problem that people had like aversion towards data system because they were hard to handle and so we decided we were going to work on 1:30:58 making an end-to-end good experience for data systems for once so the result of all that thinking and sort of like long 10-year process almost was that a whole new class of Data Systems was required and the first instance of that class is techdb seductibly is an in-process analytical data management system it does SQL and I will tell you more about it in a moment but first the first question we always get is why is it called duct TB it is because I used to have a duck his name was Wilbur he is a very cute duck 1:31:36 um duct TB is very flexible it runs really anywhere it can run from a Raspberry Pi to a huge server it has a very small footprint only 20 megabytes and it has no dependencies it's like just a C plus plus project and that's it is very feature Rich we speak a why we have a a very large sort of SQL dialect we have all the features that you would expect from a normal from a modern analytical SQL engine it has Integrations with python a lot of other languages Arrow integration and can for example directly read parquet files 1:32:14 ductibly is fast we have a state-of-the-art vectorized so-called query processing engine and that comes directly from the sort of state-of-the-art research that we did at the research group in Amsterdam for example daktibi can also automatically paralyze queries so you give it a query and it will figure out how to use all your Hardware resources duct abuse free it's free as in free beer it's MIT licensed do whatever you want build a company on top of it it's really up to you but let me zoom into a bit into some 1:32:50 things that make that to be different drdb is not a client server system and then I know that is also something where it's a bit of a departure from what people are used to all data systems Under the Sun are client server and this has been unquestioned since 1984. but Dr B is in process meaning the database runs directly in application and it means you can directly access data in the application and in fact I'm going to show you a quick demo on why this is amazing here I have a python script where I create a data frame with 1:33:20 a billion rows it's about eight gigabytes in memory and now I can spin up a ductib instance in that same python shell because it's in process and I can run a query and it will only take 160 milliseconds or so to go through these billion rows and compute the average and that has to do with yes the data is the database has a state of the art engine but it also can directly access the data in your memory of the Python process but let's zoom out a little bit how does duct TB fit on a typical architecture we 1:33:52 have our data Lake we have analysis cluster and we have the laptop with the analysts well very often the first step is that we do ETL we take the big unwieldy data set and we transform it into a bit more manageable data in like data Lake formats or parquet but meanwhile the analyst is idle and then we do the analysis project where we do explore and we try to find out something about the data and we use the cluster as well and this works really well but it creates contention with other jobs running on the cluster and it 1:34:21 slows down analysis with duct TV you can actually move the exploration and analysis on the very laptop that the user sits in front of and it has one real Advantage is there's no contention you have your own laptop it reduces latency and just generally reduces load on the cluster everyone's happy let me show you another demo here I'm now taking the dacdb either the famous New York Taxi data set it's about it's about 90 gigabytes of data it's in parque format and it's about 1.7 billion 1:34:52 rows across 50 columns and I'm just using dacdb in this shell here to you know set up The View with the trips and now I can run for example very very basic query where I group the number of very accounted number of trips per cap type and this query has to look at all this data in this in this huge data set and it finishes in under three seconds seconds on my two-year-old Macbook so that is really something that you wouldn't have thought that you can run these huge data sets on a normal MacBook right you can also have a much more 1:35:28 complicated query for example in disk query we compute the number of trips per amount of passengers the year in which they happen at a distance so this is a bit more complicated and I'm not going to expect you to read the SQL query but even this more complicated query will finish within under 10 seconds but how does duct TB fit into the machine learning space well we are not doing machine learning we are database people we're simple-minded but as you probably know the process of preparing data for analysis is the one 1:36:03 of the things that you spend most of your time on if you want to do any sort of ml problem and active actually fits really great into that process because I mentioned you can pull a parquet file you can do a lot of reshaping wrangling you know figuring out what features are required creating new features all that you can do all that in duckdb and then we have Integrations with systems like pytorch or tensorflow to directly ship the results of this reshaping you're doing to these libraries and the cool thing is that because duct TB runs in 1:36:34 process we are already in the same environment that the ml library is also going to run in which means the transfer is near instantaneous so now I hope I have given you some reasons to check out duckdb um here has some pointers but with that I thank you for your attention [Applause] [Music] there must be something in the water because llms are the Talk of the Town if anyone has tried if anybody has tried building an llm-powered application they have very likely used flank chain it's incredible that this tool that we 1:37:25 all depend on and use was only open source less than eight months ago it is one of the fastest growing open source projects out there and had more than 2 million downloads last month alone so please join me in welcoming co-founder of langchain Harrison Chase [Music] thank you Brooke for the introduction and thank you databricks for having me my name is Harrison Chase CEO and co-founder of linkchin and I'm really excited to be here so what is LinkedIn link chain is an open source developer framework for building all applications we started as 1:38:14 an open source python package in October of 2022 right in between stable diffusion and chat GPT and then we quickly expanded to have a typescript package as well we kind of came about right at the time where there was a massive increase in the number of people interested in building applications with language models and so I think we've been really fortunate to have an amazing Community that's helped build with us and on top of us and so I think that's reflected and these numbers are a little bit out 1:38:44 of date we have over a thousand contributors and so I'd also just like to take this opportunity to thank everyone who's contributed in part to Lang chain so yes yeah that's that that's the most important part to clap at so um what is link chain so I think of the value props of Lang chain in in two separate ways one we have a lot of components and these are all the building blocks that are necessary when building llm applications and so at the core of llm applications are the models themselves and so what langtune provides 1:39:20 is a standard interface for interacting with over 40 different llm providers hosters everything in that in that Spectrum um we also provide tools for managing the inputs and outputs of these llms so at the very basic level the inputs and outputs are strings but of course A lot of people are having more kind of like structured inputs so you have a bunch of different variables and user inputs and chat history and few shot examples that go into this this what ends up being a pretty complex input to this llm and so 1:39:51 we help manage all of that state and on the opposite end we have output parsers which help take the output of the language model which is a string and parse it into something that's useful Downstream because the main the main way that we see people using link chain is is not just to call a language model but to use it as part of this system and so that naturally involves connecting it with a lot of other components and we have a lot of Integrations with those other components as well so document loaders we have over 125 1:40:18 different places to load documents from and and then split them into chunks and this is very important for when you want to connect these General models to your data and enable them to answer questions about about your company your personal files we we have an integration with over 40 different Vector stores again all with a standard interface so that you can start with a local Vector store and then quickly transition to a cloud Vector store when the time when the time comes so that's one of the value adds of link 1:40:47 chain which are these little building blocks basically which you can assemble in various ways to build an end-to-end application and then on the other side we have use case specific chains and so we see a bunch of common patterns in how people are building all on applications and we put these into chains to do question answering over documentation question answering over CSV or SQL or any type of tabular data as well adding a natural language layer on top of apis so you can chat with your data that are behind apis 1:41:18 extractions so extracting structured information from from unstructured text and a variety of others and so to summarize link chain I think of has two two value props one are these individual components which you can easily use and assemble and and are in are interoptable to build your own applications and then these chains which are easy ways to get started for the rest of the talk I want to talk about three of the main areas that we're really excited about at link chain so the first one that I want to talk 1:41:47 about is retrieval and so the problem that retrieval solves is that language models like GPT only know the data that they were trained over and so that's useful and they can answer a lot of questions but when it comes time to using them to answer questions about recent data or your personal data they they can't do that by themselves and so a popular technique for for allowing them to do that is a technique known as retrieval augmented generation where you first do a retrieval step and you provide that information in the 1:42:20 prompt and ask the language model to answer based on on that data and so the benefits here are that you don't need to retrain a model and so you can use any of the the commonly available apis off the shelves and then this also helps ground the model and reduce hallucinations so it doesn't make things up so the common workflow that we see for retrieval is when a user question comes in we first do a similarity search and retrieve from a vector store various docs and various chunks of documents that are relevant for that question at 1:42:53 hand and so behind all of this there's also a separate ingestion process and this is where the document loading comes in play where you can load document from let's say your notion you can then split it into various chunks and the reason you split it is so that you can then retrieve only the most relevant ones you then create embeddings and you put them in a vector store you then pass them into a prompt and ask the language model to answer based only on that question this is a this is the standard solution 1:43:17 and it will get you about 80 percent of the way there what we've seen and what we're really interested in is advances on top of that advances that help go from that 80 percent to a more reliable 95 and some of the edge cases that pop up that necessitate these advances are things that happen when you have conflicting information or information that changes over time or when you have queries that aren't just about the semantic nature of a document but also refer to some aspect of the metadata and so we have a lot of 1:43:46 different layers on top of the basic similarity search inside of link chain to help get started with that another area that we're really interested in is agents and so the agents is a bit of an overloaded word everyone kind of has a slightly different definition the one that I like comparing agents to chains is that chains are a sequence of predetermined steps that you have coded in in code while agents use the language model as a reasoning engine to determine what actions to take and then it goes in and 1:44:17 does that action offloading any any knowledge or computation to that action it gets back in observation and then it passes it back in and so the the standard solution for an agent is effectively a while loop where you ask the language model to think about what to do you then go execute that action you get back in observation and you repeat that until the language model recognizes that it's arrived at its final answer that it's completed is subjective and so so we have some infrastructure around that and then we also have 1:44:49 various prompting techniques like react which is a paper that came out last October stands for synergize in reasoning and acting and it's it's designed to more effectively enable language models to think about what to do and take actions and so we have these as part of the standard solution and standard offering and Lang chain we're also really interested in advances in agents agents are one of the most fast moving spaces and so we're paying really close attention I'm really excited by plan and execute style agents 1:45:18 where instead of going one step at a time you do a planning step you then execute the first step in that plan and then you return to the planning step and and kind of adjust the plan from there and this helps with longer running tasks and so this was this was heavily inspired by a lot of work in the open source by by projects like baby AGI actually and things like that another area that we're paying really close attention to is tool usage so tool usage involves the language model taking an action and so this started off where 1:45:48 tools were functions that accepted a single input and that input was just string and that was because language models back in October were only really good enough to input that single string now they're getting good enough where they can input more complex structures with multiple function calls and so we're working to support that and push that forward the last area that we're really excited about is evaluation and we're excited about this because right now I think there are a lot of people prototyping 1:46:16 and there's not enough people putting things in production and I think the gap between prototyping and production exists because it's really hard to get the more complex applications reliable enough to a point where you can trust them in production and part of that is evaluation and gaining that confidence and so there's two main issues that we see with evaluation for LM applications one is a lack of data and then one is the lack of evaluation metrics you don't start with data with a lot of these LM applications you start with an 1:46:44 idea and these language models are fantastic zero shot Learners and you can get started really quickly but then how do you know how well it's doing and then on the evaluation metric side traditional ml metrics like accuracy and MSE don't quite cut it here and so one of the things that we're working on and we're excited to announce in a few weeks is a platform to help with this help with enabling both the collection of data as well as the evaluation of it and evaluation of it comes in a few different forms there's 1:47:12 there's traditional metrics but it's also just making it easier to visualize and understand what's going on the most common way to evaluate is the vibe check which which sounds bad but it's actually it makes sense that that's the best way because it's really complex and allowing people to gain that insight as to what's going on is important and so we want to help with that as well as as well as more advanced metrics we also have a ton of Integrations with databricks so we support the the LOM 1:47:41 endpoint we support ml flow and we have an integration with spark both chains and agents where you can interact with your data and the best part of all of this is that you can run link chain from inside data bricks and so that leaves me with the question we're here this is this is a generational moment to build things with ML and Ai and so I hope that that you guys take advantage of Lane chain take advantage of databricks and and what are the amazing applications that you guys are going to build I'm 1:48:12 excited to see and and please share them so that's all I've got thank you guys for having me have a great day [Applause] [Music] thank you thank you Harrison Lang chain loves databricks and databricks loves Lane chain the databricks assistant which is live check it out was built using light chain in order to power these llm applications though we need a framework to build our language models and according to papers with code the most popular framework to do that with today Pi torch for our next speaker we have the former 1:48:57 head of Pi torch who's going to talk about the past present and future of AI for Developers so please join me in giving a warm welcome to Lin Chao CEO of fireworks [Music] [Applause] [Music] hi everyone I'm Lynn today I will talk about developer Centric AI the past present and bright future I have always been passionate about building awesome tools giving to developers to do the best in their jobs when I started my career I built one of earliest in memory coloring databases fast document store with inverted indexes and many data 1:49:52 tools by logging metrics experiment platform and so on and so forth I moved to AI about six years ago seeing a huge surge of data volume driven by AR workload a LED pytouch team at meta accelerate AI researchers in open source and bring huge production workload in production at metascale after having done all this I started fireworks on AI with link the mutual and awesome group of Engineers continue our journey to bring awesome tools to broader set of developers here including all of you in this audience and help you to be the most equipped 1:50:33 proliferative creative in this new AI generation fireworks aim to accelerate product innovation building on top of dnai we offer experiment and production platform where you can get access to the most data art research of Open Source models bring your data and use our previewed tuning evaluation recipes customize your model deploy easily into our inference Services optimize for cost to serve and then we free you up to focus on your product Innovation ideation and iteration when your product idea is ready to deploy full speed full-scale to 1:51:20 production you continue to stay with our inference service and go from there before I talk about more details of products and why us let me start with where AI Journey began Ai and deep learning is our new not new concept more than 30 years ago yellowcon as one of the pioneers of deep learning published a paper in 1989 back popular propagation applied towards handwritten zip code recognition the model still reads very modern today so on that point for many years AI remained as a research topic researchers need a maximum amount of flexibility 1:52:05 to innovate and maximize model quality most efficiently towards the Trinity that they have when we started pytorch was started for researchers to export and invade faster and we observed around every three years major model architecture disruption happens and every few months significant incremental Improvement on model quality happens the pace of value created by research has a Slowdown but accelerated in recent years what is pytorch you can think about it as numpy for AI accelerated by gpus tpus your accelerators of course it runs on 1:52:50 CPU as well pytorch use model as code philosophy for dynamic neural networks including Dynamic shapes control flows those are essential for air researchers to disrupt building disrupting building blocks in AI why model as code is so good because programming first program first structure is the most open and composable as opposed to rigid closed systems with hidden functionalities many research institutions including open AI switch over to use pytorch a few years ago for its flexibility to accelerate their Innovation speed 1:53:34 so many researchers become entrepreneurs or move on to deploy models in production production deployment shift the focus on system performance stability versioning scalability cost to serve and so on as you can see there are quite natural tension across these two cohorts of air Developers researchers naturally gravitate towards breakthroughs and break current Paradigm to new ones and production developers optimize for current paradigm pytorch continue to speed up researched production transition by ease the tension in between 1:54:13 and that's my focus in the past six years one of the most important use cases we innovate on is ranking recommendation because it's everywhere in our Digital Life python release recommendation libraries last year driving more than 10 times performance Improvement and memory utilization reduction it makes the large complex reconditioned model practical to be deployed in production we continue to to bring more awesome models on mobile devices which is more resource and power constraint it requires order magnitude Improvement 1:54:54 of reduction of python time size and latency Improvement as you can see there are many cool features we enabled through this on mobile products pytosh is today deployed very broadly in production driving more than 50 trillion influence per day on servers in more than 70 billion in first per day on mobile phones pytorch application industry has penetrated across many domains from content standing of language vision speech to a complex task of ranking recommendation robotics to a high Precision task of medical cancer 1:55:40 diagnosis treatment drug discovery many variations of autonomous driving vehicles in package build a Vibrance Community as we can see from the interpretation of archived people covering both research and production a constantly growing over the past years towards a dominant position as Janai is the most interesting topic in this Summit most of gni models are written in pytorch so we are actually going through a very interesting model development paradigm shift now from before where a lot of individual developers or institutions are spending 1:56:28 lots of Energies collecting curating cleanup data and training a particular model architecture from scratch and then deploy into production to now you can easily customize your model using much less data as you can see from this Paradigm from bottom up there are many Foundation models already pre-trained and the race released based on public data available on internet and then you can specialize those Foundation models with your domain specific knowledge for example for finance legal health care and so on with their custom with their company 1:57:06 proprietary data then you can further customize towards your company specific product by building your mode and the further along you can even provide a personalized model for example building avatars or air assistance to make your product even more interesting so we have a foundational shift right now because of Jenny I in AI development you still get the state of our models from research but with much less data you can start to customize and build interesting products and do product exploration much much faster 1:57:50 one of out of many product ideas you tried getting to full-scale production and your Production Drive more data pushes better model quality improvement and you experiment and explore more product ideas in this quick iteration cycle is new as you can see in AI developer cohort we have a growing a new growing product developer coming up like many of you sitting in this in this hall right now so fireworks is created to accelerate product Innovation building on top of jnai by solving many of the challenges we heard from many of you 1:58:34 first of all the challenges is there's a huge amount of dynamism of General models and there's a start competition of many model providers constantly improving model qualities one after another it is very concerning to many people how do how do I manage this ever-changing air landscape and which model should I use and stick to as you can see from the right hand side of the diagram with increasing timeline the intensity velocity of model Innovation accelerates second business task is never simple it usually is a complex business 1:59:15 workflow that needs to be decomposed into one or multiple air models and which layer in this hierarchy do you interject and there are many interesting business problems people coming to asking us how to solve that for example I want to use natural language as the interface to retrieve precise information from my company internal documents or precise information from my product catalog or I want to automate my construction workflow to minimize timeline and minimize cost or I want to create diagram charts at Scale based on my data 1:59:51 this is the first mile problem and when people get slowed down not knowing where to start more specifically within each model family usually there are different model sizes being provided from small medium large sometimes to extra large and you can choose to zero shot access the base model or prompt tuning to get better quality or fine-tuning user data to get even better quality for specific tasks you can actually fine-tune a equivalent quality or better model with a smaller size so which one would you choose of course AI January AI is expensive 2:00:31 because it's computational and memory intensive in the past two to three years AI research has been pushing for bigger and bigger model size until chinchilla law get established that there's optimal token to parameter ratio about 20 to 1 but still most people are using around 7 billion to 13 billion model parameters and that's much more bigger than traditional machine learning algorithm for example of XG boost many companies can slow down or stuck because of the um the intensity of the computational cost 2:01:06 and quickly run out of pre-planned Hardware budget so we think the competition in the JNL model space is actually great because the party developers are machine learning Engineers can write a free wave of ever Rising model quality there will not be one model provider dominating the whole Space there will be many of them keep advancing the quality altogether fireworks want to enable you to continuously testing and evaluating the best model to incorp into a product yes continuously because the Cadence of change here is extremely fast 2:01:52 second cost so we love this because we have been driving customer service performance in general performance optimization for python for many years at hyperscalers we hyper optimize the cost for fine-tuned models fireworks today offer at equivalent model size five times cheaper cost compared with open Airbase model and 30 times cheaper cost compared with their fine-tuned models that's a whooping cost reduction we also quickly optimize latency for open source models for example a Nissan one that we deliver more than 10 2:02:38 times latency reduction based on the original implementation this requires deep hydrogen time optimization and integration with the best kernel as well as carefully crafted scheduling orchestration design and optimization all coming together we also understand to accelerate your product exploration you need to do a lot of concurrent parallel experiments at the same time and oftentimes with increasing number of experiments it increases your memory footprint as well that means you have to pay more we want to take away that concern for 2:03:16 you as you can see we can effectively control the memory footprint with increasing degree of experiments that means with fixed budget you can run a lot of more experiments at same time in your time to Market will significantly be reduced tying all this together fireworks provide a rapid experiment in the production platform we give you free access to the state of art research with your own data you can use our pre-built fine-tuning and evaluation recipe you can change it adjust adjust the W As You Wish run the 2:03:50 fine tuning on-prem or on cloud as you wish and their custom model will be deployed to our general AI inference service with minimum cost to serve you can then focus on your product iteration and exploration when you like your idea and want to grow into full production scale do that within the inference service without changing to anything else today we build our inference service on top of gcp AWS and quarries we plan to expand our cloud provider partnership quickly down the road our API is fully compatible with open 2:04:25 air API and lynching so if you you don't need to change your code much to use our services if you feel the challenges I just mentioned resonate with you and also want to know more about fireworks find me after talk and also find an awesome fireworks AI team we can give you a demo and give you more information about worldwide building also stay tuned for many more exciting announcements coming up soon thank you looking forward to seeing all of you [Applause] [Music] our next speaker is a Pioneer in the field of computer vision 2:05:17 he was formerly the head of AI research at Facebook in Menlo Park and as a distinguished professor at UC Berkeley in his talk he's going to talk about the latest developments of modern AI applied to robotics please join me in welcoming jitendra Malik to the stage [Applause] hello so I want to talk to you about the sensory Motor Road to artificial intelligence but first we should know something about natural intelligence and natural intelligence to make sense of it we should think of it in the light of evolution so something 2:06:03 like 540 million years ago we had the first multicellular animals that could move about and moving was great because that means that you could get to food in different places but to get to food you needed to know where the food is which means meant you needed Vision or some form of perception and these two abilities go together and there's this great line from Gibson we see in order to move and we move in order to see and so if you think of the brain or the nervous system of a fly or a fish or a frog it's basically this 2:06:39 connection between perception which could be Vision it could be hearing to action and action is moving about in this case so if you zoom ahead to like five million years ago which is the when the first hominid split off from primates you have this additional accomplishment which is that we start to walk on two feet which means that the hands are free to build tools ah make tools uh and and then you get the Advent of dexterous manipulation and planning and all the rest of it and then the last big development is of course for modern 2:07:15 humans like hundred thousand years ago or something which is when we have language which is uniquely human and abstract thinking and symbolic Behavior but what's important to keep in mind is that most of the brain is devoted to perception and action and connecting the two and if you think of the entire evolutionary history as being 24 hours of intelligence languages in the last two or three minutes that's all it's very important but it's only the last two and three minutes and we in this audience I don't need to 2:07:50 say much about language models but it's they're incredible and they can do the I'll pick one line they can pass the bar exam at 98th percentile incredible so given such amazing accomplishments what I want to connect you to is the fact that we have so much trouble on another side of AI which is the robotic side of AI or self-driving cars I have been in this field for 40 years we have had self-driving cars for 30 years cars which drove across Europe from Berlin to Paris and across America and so on and there's been a lot of hype 2:08:28 about self-driving cars and we'll get them we'll have the cars but think of how hard it has been and it's something which so we can pass the the law exam which takes years and something that a high school kid of 16 wait after 20 hours of training is good enough at we are having trouble with this I can make this problem even harder so think about what a 12 year old kid can do right a 12 year old kid in a kitchen with knives and forks and ladles and so forth can do all these kinds of tasks and no robot today can do all of 2:09:06 these tasks okay this is incredible something easy we can do and this is something that people in AI have known for a long time it's a paradox right a law exam the bar exam which takes years of study is hard and cooking an omelette it's supposed to be easy but actually it's the other way around marwick had this great line which is that things like chess are easy language is easy what's hard is what a one-year-old can do and Steve Pinker has a beautiful line for this which is that the hard problem what we have learned from years 2:09:48 of AI researchers that the heart problems are easy and that the easy problems are hard and as the and then he goes on to say that the gardeners receptionists and cooks are secure in their jobs for the years to come and why so the question is why and model X argument was to do with reverse engineering and it was in the area of Designing Ai and there the question was okay what we have what has emerged through hundreds of millions of years of evolution is much harder to sort of reverse engineer I think actually the argument is slightly 2:10:25 different we know how AI has been achieved it's largely been through uh deep learning applied to huge amounts of data and the kind of data that we have what's the kind of data we have we have huge amounts of data for language models why because everything is on the web right all the books are on the web Wikipedia is on the web read it for on the web GitHub is on the web this enables you to train these these models so all this data is available explicitly think of what's the data needed for sensory motor training 2:11:03 you need to know what images I take and what are all my muscle commands and what are my neural activations hey that's pretty personal I'm not uploading that on the web okay we are not going to get all that data in huge amounts on the web we could get parts of it we might see what the images are but we're not going to get all of this and therefore we will need new clever ways of solving this problem it will need AI it will need learning and I'm going to take you through a little bit of how we 2:11:33 can do this and I work a lot in robotics and this is one of the robots that we trained in my group and it's solving problems its footwears stuck against The Rock and it managed to go through uh this robot by the way is blind it has no vision it's okay and you see it walking downstairs it's not even aware of the stairs but it manages to stabilize here's another example which is on lose mud pile and at a construction site and so on and so forth so you need a lot of Versatility so these problems are 2:12:23 actually hard and let me think of it more formally uh computer vision is like pattern recognition and we the basic challenges generalization so we can't have a formal definition of a chair but we can train a classifier for chairs by giving lots of examples when we come to these motor control tasks it's a different game one part of it is handled by classical control theory which is robustness to disturbances you do feedback control there's the second part which is which is actually the more interesting or 2:12:57 harder part which is that adaptation to these different conditions so I showed you this setup where this robot dog has to walk in all these different terrains so adaptation that's very important so we can use classical control theory techniques to Train control controllers for particular situations and for example Boston Dynamics has a lot of work demonstrating these kinds of controllers but where Ai and machine learning can come in is can we build one policy to walk in all of these situations one policy which figures out 2:13:36 automatically which situation you are in and then it works in them and this is a work from my group called rapid motor adaptation where we figured out essentially how to do this and I'll just take you through the big idea so the big idea is we train in this robot in simulation and this base policy is like figuring out how to change all the joint angles and things like that and there is this variable Z which you can see in that diagram and this Z captures some aspects of the terrain as some low dimensional vector and 2:14:14 if we knew the Z we have this policy which will do different things so it does different things in sand versus and hard ground and so on and so forth okay but we how do we do that in in the real world we need a way to estimate that itself and for that we sort of need to go meta so we have an adaptation module which looking at the past history figures out how to uh what does he must be so the intuition is something like this that when I walk if I walk on hard ground I perform certain actions and then there are consequences of those 2:14:51 if I do the exactly the same thing when I'm on a beach it's going to be different because when I put my foot down and I try to lift it up it's not going to lift up so easily and that kind of signal from the state and actions over the past one second or half a second I have the signal and uh and and that's basically it I mean there are some details obviously but I'll show you an example here so here is an experiment which uh my student Ashish did where he's pouring olive oil and on this waste of good 2:15:24 olive oil on a mattress and then he's going to take the robot and if you look at the legs of the robot he's got like plastic socks to make it hard and then he's going to make this robot walk and what happens is that it it starts to slip right let's do it in slo-mo so what's happening is that it has some estimate of the extrinsics this Z vector and that estimate is wrong because it's slippery now what should happen is that over time when it walks it estimates this and that's being done by this adaptation 2:16:03 module and once that estimate comes through it works out and then it's recovered I'll give you another example now it's a much harder problem I've got this robot which has got a vision system in it and it's going to walk in much more treacherous Terrain so now there's a camera and it's everything is on board it has no Advanced knowledge of the terrain and it's using similar techniques notice that this robot is a very short robot compared to the heights of the stairs and it's yet able to manage in 2:16:48 these conditions and it has no prior knowledge thank you and this is slippery ground as well as a slope so and these ideas apply to other applications besides walking thank you thank you and there's one which is dexterous manipulation so if I want to cook in a kitchen I need to be able to manipulate with my multi-fingered hands and here's an example of that similar idea estimate what the situation is the so in this case there are objects of different size different shape different weights and and I mean that's what this says and 2:17:42 these are being estimated online and uh so for example the shuttlecock is very light it's only five grams okay and some of the objects are heavy okay okay there's this empty bottle a Rubik's Cube very importantly it's exactly the same policy the robot is blind it has no prior knowledge of what it is trying to control but only from the appropriation what it feels in the fingers it is able to do the right thing and I have more examples here so so I think that this is the future I think we have to machine learning and AI 2:18:27 is essential for the success of Robotics because we need flexibility and we need adaptation but I want to conclude in the last two minutes that I have with some general philosophical remarks so I entered the field of AI 40 years ago for me the success in the last five years is incredible I would never have thought we would have get got so far in that what we did in the last five years where we were five years ago versus now and broadly speaking deep learning in the last 10 years but how do we do the rest I think 2:19:00 robotics is very important sensory motor control is very important without that we have not achieved intelligence there are these ideas for this which go back to Alan Turing Alan Turing who's like the father of computer science in a way and he has this paper from 1950 which has the Turing test but it has this great line instead of trying to produce a program to simulate the adult mind why not rather try to produce one which simulates the child's so it's essentially a program of learning but learning with stages the way children 2:19:31 learn and we not know a lot from our psychologists colleagues about how children learn children go through various stages of learning there's this multimodal stage this kid in the Crypt she's playing around she's poking at objects so she here sees it she hears the sound she puts things in her mouth her motor system is being activated all of that is training data all of that is being used the kid does experiments so Allison gopnik has this book called The Scientist in the crib so when this kid when your toddler is being 2:20:09 difficult and throwing food down you should say she's actually a scientist in the crib she's conducting an experiment from which is building models of the world around us and and this is very important and then finally this child at the age of two you take them to the zoo you give them one example of a zebra it works and our programs we need to give them thousands of examples of zebras and uh and then at the age of 16 we give them 20 hours of training and then they can they can drive right so we I personally believe that 2:20:43 this developmental story is going to be needed for all of AI and the psychologists have actually told us what these steps are multimodal incremental physical explore be social learn from others and finally use language so language is very important but it is in a way the crowning achievement of intelligence and it should be built on the substrate of physicality thank you very much for your attention [Applause] please welcome to the stage co-founder and Senior vice president of field engineering arcelon tavakoli 2:21:26 [Music] good morning everybody it is my distinct pleasure to help Host this fireside chat today we've clearly been spending a lot of time talking about AI you know where is it going what's the future how do we get here what's next and my guest this morning is somebody who has been living in that world for a very long time and needs very little introduction please give a warm Round of Applause to welcome the former CEO of Google Eric Schmidt foreign [Applause] [Music] hi everybody good to see you thanks for coming 2:22:24 I'll be honest I've been at databricks for 10 years now and uh my parents were the most excited been that this morning they're like they didn't care about anything else you'll be on stage with Eric Schmidt I was like yes well you know I was here before I was here 30 years ago on this stage and I introduced Java right here yeah so quite an accomplishment do do the math so there's obviously a lot of directions uh that we can go into I'm pretty sure I'm gonna get yelled at to get off stage 2:22:59 before we can get through everything but you know one of the topics that everybody is interested in is just the speed and fervor which things have come on the generative AI side and you've been living in that world for a while now so curious to just hear your perspective why now What's led to this kind of interest in Drive well what's the value of inventing a new kind of intelligence foreign probably pretty high what's the value of improving every single business process Communications process entertainment process 2:23:34 educational process in the world pretty high how big I did a report for the US government which said it was a 40 trillion dollar business that's big enough to get everyone to play and the other thing that's interesting is our industry when I started which was almost 50 years ago we were not very good at talking about ourselves we were actual nerds we were not actually nerds with shine if you will and the industry has changed and we've gotten really capable of hyping ourselves to the max right and I can 2:24:13 give you the data of adoption look at the rate at which chat GPT was adopted versus Gmail as a simple metric but the fact of the matter is it's both better hype better excitement and compression of time one way to understand this is that before the internet existed your company you actually had to sell you had to call on doors and send CDs and so forth and so on and the internet created at least for digital Goods an immediate instance success history which is now what's driving this there's this sense 2:24:45 that you can go completely non-linear with something successful the other problem is that the compression of time means you have almost no time to get it right and small differences in timing can have huge statistical outcomes based on when you started and when you got there which is why everyone is in such a rush to get funded to get people and so forth everybody kind of understands do it right now okay so there's a there's a little bit of that now it's very interesting somehow you know generative AI all of a sudden 2:25:21 has become all about the model everybody talks about the model and um this big question we get asked a ton about it from customers it's been a topic where I think it's fair to say for many generative AI became synonymous when chat GPT and open AI came out and so uh there's this debate is the world going towards this part where there'll just be a small handful of these large foundational models or is it going to be more towards whether it's open or specialized model is it a balance we get asked a ton obviously uh you know I've 2:25:56 seen you talk about it how do you think about what the future is the need of the different kind of types of models and how they'll be leveraged so about a month ago a number of us wrote a blog post which tried to answer this question yeah and the answer of course in Tech is both you're both going to see these incredibly powerful very very flexible what we call Frontier models and they're going to get used in business processes but you're also going to see incredibly powerful specialized models both will happen for different 2:26:28 reasons if you think about it as a business so one of you is trying to build a company that uses advertising but you have an idea to generate the ads for the customers instead of making the customers generate the ads that's a good idea and by the way Google has just now announced they're going to do that good job so you do that do you really want a fully functional learning changing and so forth powerful general purpose model or will you be better off served with a specialized model that does a better and 2:26:59 better job and I think in that case you would want the latter if the questions are more open-ended you're probably going to want the more General model since we haven't proven the business models of either these right we literally don't know exactly how they're going to get monetized yet which by the way is a testament to the world's Financial system that we could raise billions of dollars without having a product plan a revenue plan a product price and an identified customer thank God for the financial system and right 2:27:30 we're all here because of that so so I think you're going to see both um and one of the most interesting questions is the rate of diffusion from the frontier models to the open source models so if you take a look gpt3 is now the functionality is largely available in the equivalent of alpaca um that's roughly two to three years in terms of diffusion from a very expensive very powerful model to something which is generally available and effectively free yeah and I think the question becomes as we keep getting more and more powerful at 2:28:05 what point is it good enough like for for the different uses well the correct answer is it's never good enough in software yeah right we still we have the M1 the M2 the M3 the M4 we have five nanometers five three nanometers two nanometers it's never good enough but somehow we find a way to use up all of the the software it's the old rule you all probably don't remember this it was a Grove giveth and Gates taketh away right was the saying that Intel would add CPU yeah and Microsoft would immediately use all the all the hardware 2:28:38 for the software that they were building on top this is you know 30 years ago it's still true okay um so we were talking about this you know briefly backstage there's a lot about the models we spent talking about the specialized models and all that um in this world how do you think about the importance about things such as like data and governance and data quality in terms of driving improvements and Effectiveness in the use cases around it yeah what's interesting is if you look across the fields that are using AI 2:29:11 people people talk about the algorithms so if you think about it you need what do you need you need Hardware you need scientists you need programmers who can build scalable systems which is not the same thing as scientists and you need lots and lots of data in many fields the most interesting data is data that is being synthesized right so for example if you have a physics model and you can act you know the physics is the physics you know physicists are always right in in my in my world they can essentially 2:29:38 simulate and therefore generate the data that's needed for training there are plenty of other examples where people will do essentially sampling of the data so they'll for example sample a series of Q of questions and answers and then they will perturb and generalize those for training all of these tell you that data is the key and it's important that the data be curated you I wish that you could just sort of run these things over every piece of data regardless of structure and so forth and I think this 2:30:08 is sort of why you guys founded databricks as you were trying to do this right and you've done it super well so you're one of the components you need to also be helping people hire the right technical people get the hardware and so forth to make it complete got it and so in the role of that you you basically just touched upon it of data right now the other aspect you talk about how important will human feedback be in looping that in in order to basically make sure the models are improving and AI is successful ultimately overall well 2:30:38 rlhf um is a really cool idea when I first heard it I figured it wouldn't work but it turns that it works really really well and there are now newer new techniques which allow our lhf plus Laura basically quick adaptations of the model which allow you to fine tune using relatively standardized open source packages that are available on GitHub and so forth so that has led to this enormous explosion of variance and so the most likely scenario in my opinion we'll see is you're going to get a pre-trained model that's quite good a 2:31:16 base model and then you're going to see every conceivable combination in the pipeline right rlhf various other things synthetic data synthetic training evaluation and so forth to get there the next thing that happens after that is the ability for these models to call to some reference point some ground truth source and there are startups that are working on calling when it gets confused which they always do calling out of the model into something to get a reference point and then put it back okay um maybe maybe Switching gears a little 2:31:50 bit slightly from that we've been talking a lot about the potential the opportunities for AI uh there's definitely a lot around there I don't know if fear is the right word but uncertainty of where it's going and that kind of coincides with discussions about regulation and what's regulation going to be how things are going um how do you see you know do you see us broadly getting alignment on how to regulate Ai and the path forward or at least the challenges that are there categorizing those and what we should be 2:32:17 how we should be thinking about those going forward well it's interesting that every single politician that I speak with every single leader I talk to is now an expert in Ai and they know nothing about what you all are doing so maybe that's always been true but it's sort of alarming to me and so the first thing I always tell them is um do you remember that movie where the robot gets out and then the female scientist slays the robot at the end of the movie that's a movie okay that's not we're not building Killer Robots yet 2:32:51 right and that usually sort of disarms them so when you talk within the technical people within the industry who are I think the only people who really understand what's going on there's a consensus around three reasonably clear short-term threats that are important the first one is the one around misinformation and disinformation and we can talk a lot everyone understands what that is and everyone understands it for the more generative AI is going to be used enormously for that kind of stuff so we've got to talk 2:33:20 about that the second thing is the ability for these systems to do various forms of Cyber attack and the ability for these systems to do various forms of Bio basically bio threats of one kind or another and the consensus is that today so first place AI alignment is a term where how do we get these AI systems to follow human values AI safety means what are the guard rails that we put on these systems if you look for example a gpt4 open AI spent about six months they had a whole team who basically using rlhf and other techniques constrained the 2:33:57 model yeah and those constraints apply both at the API level as well as at their app level so people kind of understand that model and the and there's always this question of how do you how do you test for safety issues that you don't know yet without deploying them so there's a general fear that these systems when they're when they're launched will not be fully tested because they'll have some emergent behavior that we have not yet seen and therefore we can't test them so that's problem number one 2:34:26 problem number two is that because of synthetic biology and so forth it's likely that these systems can accelerate at scale evil people and so if you sit down with Google long enough and you understand enough about biology you can probably get to a bad pathogen these systems make that somewhat more likely so we have to sort of think about that so the governments around the world all have variants of approaches to this but the simplest way to frame it is there are scenarios of extreme risk and these systems are going to get regulated 2:35:01 around extreme risk I'm not talking about the things we always complain about about you know you know Johnny Johnny's dog ate you know Susie's homework kind of thing I'm talking about thousands and thousands of people harmed and killed from something yeah so uh you know maybe one or two more questions for me because I know we'll run out of time but it always happens we've I don't I don't want to call them kind of uh hype Cycles but whenever we get these technology there it seems like we go through 2:35:33 transformational technology there's some fear there's some people who fear it there's some people who believe that tomorrow all of a sudden we don't need programmers anymore because generative AI is going to take up neither the two actually come to fruition um what's your sense of the timeline for when we will start seeing very very meaningful we're already seeing some of it but like kind of almost crossing the cast of meaningful adoption and leveraging of generative AI like kind of across all Enterprises and spaces so so 2:36:01 the first use of generative AI is already with us which is programming and the first and most obvious use is in enhancing the power programmers and this shouldn't surprise you because every generation that I've been through that technology was first used by its inventors to make them more productive by the way I'm old enough this is what email was invited invented for right this is what we used Unix for way back when we used it to make ourselves more productive and I used to tell people inside the various companies that I ran 2:36:33 that why don't you start by making yourselves happy which is really hard and then why don't you make your friends happy and then come back to me right that sort of works we sort of forgot that we decided that we could build arbitrary consumer products without actually using them ourselves and that's a mistake so I think the first use is programming the gains in programming are profound and it looks to me like half of programming can be sort of think of it as a doubling of productivity right and that's going to that's going to continue 2:37:02 there's a whole bunch of startups that have even more sophisticated ideas around that I think that's the first one I think the second one is going to be in doing things which are in the normal course of businesses I earlier mentioned this notion about advertising why again why do I have to decide and argue through my tweet why can't the computer ask me what I want to talk about and generate a tweet that is guaranteed by its metrics to be the most viral right so if if I'm trying as a marketeer to 2:37:34 have the biggest impact surely the computer working with me can help me make it happen so I think that all all of these systems where the compute the human is doing something that is against some goal that's administered by computers is probably the next one makes perfect sense to integrate that makes sense right we know what virality is because this is what we do all day whereas we do it 24 hours a day and you're sleeping yeah so we'll tell you how to do it and and again that's an improvement in efficiency 2:38:03 quality and then the next things are these more specialized markets we don't yet know what the value of intelligence is we don't know how to price it but it's obviously High okay I like I like that set of categorization and we see something similar in terms of the use cases people are adopting now one of the big barriers right now is people think about you know llms and building them and using them has just been costs cost is a huge barrier it's what you know the cost to train them access the gpus no secrets why you know 2:38:36 Nvidia is now a trillion dollar company uh you know I'm just curious how do you think that barrier of cost both for training for inference for Access for infrastructure you know and access to the models coming down over time to make them more accessible it's it's interesting that the that the training cost is extremely high and going up by orders of magnitude there are people who believe that the frontier models which cost on the order of 100 million dollars to train plus or minus will go to a billion dollars to train one of these 2:39:05 things so that is a massive change from a software perspective in my career we've never seen that kind of price increase to do software which is what they are on the other hand inference that is the the ability to actually answer the question looks like it's going to become trivially expensive in other words incredibly inexpensive and so you end up in a situation where you train one of these models and your biggest problem is making sure it doesn't get stolen right you spent a billion dollars for these 2:39:33 weights right which can basically be put on the equivalent of a hard drive you need to make sure they don't get stolen and used by your competitor or or whatever and so my guess is that it right now you have Nvidia leading it's it's a sore point for me because at Google we have this whole TPU strategy but 95 of this of the training is being done on Nvidia a100s and now h100s and Nvidia has recently talked about a successor next year uh which they cleverly call h100 next which sure looks like it's another 2:40:07 acceleration in power yeah so what Nvidia did particularly well is they worked on what is called the fp8 cluster where basically it's 8-bit floating Point multipliers very quickly and they put a special module in that that helped them and now they in the h100 they have an integrated memory architecture that's stronger one way to understand that is that the the way this training works is you have this very large amount of data that's larger than you could put on on a computer it's essentially in the network 2:40:35 on the in the data center and the algorithm is going back and forth and back and forth and back and forth for hundreds of millions of dollars yeah anything you can do to decrease Network latency bring the memory closer is a huge speed up we're still in the phase where these computer architectures are evolving faster than I've ever seen much faster than Moore's Law much faster than I saw CPU architectures because of this unique nature of these llm architectures and that's going to continue for a 2:41:04 couple more Generations okay um we're about out of time one last question you know you've got tons of Enterprises uh sitting in the audience here if there was one peaky key piece of advice you could give them of this is what you need to think about in order to be successful and successful in harnessing the power of gender of AI what would that be well the small companies are run by people who are here because you understand that this is core to your business so a simple rule for a small business is you're not going to be 2:41:32 successful unless you use AI in your model and a simple a simple way of thinking of your business is you have an Android or an iPhone by the way Android is more popular than the iPhone just for data and you have a network and then you have a fast you have a server typically in the cloud somewhere which is using AI in your business yeah so think of it that way you understand all of that for larger corporations and I talk to lots of the leaders of those what I tell them is the following you don't know what you're doing yet in 2:42:02 this space I say this in a nicer way you don't know what you're doing in this space so and your team may not either so give them the following assignment for every business unit you have come back and show me a prioritized list of what you can do with generative AI all right just show me the list right and get them all together and most companies that have done this will come up with 400 or 500 ideas some of which are customer service some of which are Revenue enhancing some of which are security right and they'll come back and 2:42:35 then the CEO says holy crap right look at all this stuff that I should be doing and then they realize they have no people to build it and then they have a crisis awesome well Eric I'm pretty sure I speak for everybody here it says thank you so much for making the time for this it was amazing congratulations to you guys and congratulations for everybody thank you thank you our salons again appreciate it sure thank you okay so one last announcement for folks uh this wraps up you know basically today's day of Keynotes a bunch of 2:43:10 interesting really interesting breakouts save the date data and AI Summit basically 2024 will be back here at Moscone June 10th to 13th as far as I know we're the only conference scheduled that week if that changes don't look at me it's not on us right take care everybody cheers