{ "cells": [ { "cell_type": "markdown", "id": "30eb1704-8d76-4bc9-9308-93243aeb69cb", "metadata": {}, "source": [ "## This demo app shows:\n", "* how to use LlamaIndex, an open source library to help you build custom data augmented LLM applications;\n", "* how to ask Llama questions about recent live data via the You.com live search API and LlamaIndex.\n", "The LangChain package is used to facilitate the call to Llama2 hosted on Replicate." ] }, { "cell_type": "code", "execution_count": null, "id": "1d0005d6-e928-4d1a-981b-534a40e19e56", "metadata": {}, "outputs": [], "source": [ "!pip install llama-index langchain" ] }, { "cell_type": "code", "execution_count": 1, "id": "21fe3849", "metadata": {}, "outputs": [], "source": [ "# use ServiceContext to configure the LLM used and the custom embeddings \n", "from llama_index import ServiceContext\n", "\n", "# VectorStoreIndex is used to index custom data \n", "from llama_index import VectorStoreIndex\n", "\n", "from langchain.llms import Replicate" ] }, { "cell_type": "code", "execution_count": 2, "id": "d9d76e33", "metadata": {}, "outputs": [ { "name": "stdin", "output_type": "stream", "text": [ " ········\n" ] } ], "source": [ "from getpass import getpass\n", "import os\n", "\n", "REPLICATE_API_TOKEN = getpass()\n", "os.environ[\"REPLICATE_API_TOKEN\"] = REPLICATE_API_TOKEN" ] }, { "cell_type": "code", "execution_count": 3, "id": "75275628-5235-4b55-8033-601c76107528", "metadata": {}, "outputs": [ { "name": "stdin", "output_type": "stream", "text": [ " ········\n" ] } ], "source": [ "# You.com’s Search API is used to augment LLM responses with the snippets that contains rich info on the query results\n", "# To use the You.com Search API, you can email api@you.com to request an API key. \n", "YOUCOM_API_KEY = getpass()\n", "os.environ[\"YOUCOM_API_KEY\"] = YOUCOM_API_KEY" ] }, { "cell_type": "code", "execution_count": 4, "id": "c12fc2cb", "metadata": {}, "outputs": [], "source": [ "# set llm to be using Llama2 hosted on Replicate\n", "llama2_13b_chat = \"meta/llama-2-13b-chat:f4e2de70d66816a838a89eeeb621910adffb0dd0baba3976c96980970978018d\"\n", "\n", "llm = Replicate(\n", " model=llama2_13b_chat,\n", " model_kwargs={\"temperature\": 0.01, \"top_p\": 1, \"max_new_tokens\":500}\n", ")" ] }, { "cell_type": "code", "execution_count": 5, "id": "effc9656-b18d-4d24-a80b-6066564a838b", "metadata": {}, "outputs": [], "source": [ "# chat about live data by first making an API request to the You.com search API\n", "import requests\n", "\n", "query = \"Meta Connect\" # you can try other live data query about sports score, stock market and weather info \n", "headers = {\"X-API-Key\": os.environ[\"YOUCOM_API_KEY\"]}\n", "data = requests.get(\n", " f\"https://api.ydc-index.io/search?query={query}\",\n", " headers=headers,\n", ").json()" ] }, { "cell_type": "code", "execution_count": 6, "id": "8bed3baf-742e-473c-ada1-4459012a8a2c", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " \"hits\": [\n", " {\n", " \"description\": \"A two-day virtual event focused on AI and virtual, mixed and augmented realities.\",\n", " \"snippets\": [\n", " \"About Meta Connect\\nWho should attend?\\n

Everyone is invited to join us virtually for Connect, where you will:

\\nIs there a cost to attend?\\n

Connect is free! You can catch all the content on Facebook. Participating in Meta Horizon Worlds will require a Quest device.

\\nWill sessions be recorded?\\n

Yes, select sessions will be recorded and available for on-demand viewing after the event.

\",\n", " \"Expanding reality, today and tomorrow\\nJoin us virtually September 27 - 28, 2023\\nWays to watch\\nJoin us virtually.\\nWatch the livestream\\n

Connect\\u2019s Keynote will air live on Facebook @Meta.

The Developer State of the Union, developer sessions, feature launches and more will air live @MetaforDevelopers.

\\nVisit us in Meta Horizon Worlds\\n

Explore a world inspired by Meta\\u2019s Menlo Park campus. Experience Mark Zuckerberg\\u2019s Keynote in 3D. Earn exclusive rewards and even play Meta Horizon Worlds experiences like Super Rumble with other attendees in Worlds.

\\nFAQ\\nGot questions? We\\u2019ve got answers.\\nChoose a category\\nAbout Meta Connect\\nWho should attend?\"\n", " ],\n", " \"title\": \"Meta Connect 2023 | Join us virtually September 27 - 28, 2023\",\n", " \"url\": \"https://www.metaconnect.com/en/home\"\n", " },\n", " {\n", " \"description\": \"Explore Twitter's latest discussions on meta connect\",\n", " \"snippets\": [],\n", " \"title\": \"meta connect on Twitter\",\n", " \"url\": \"https://twitter.com/search?q=meta+connect\"\n", " },\n", " {\n", " \"description\": \"Meta's annual Connect conference started today, and this means lots of new hardware and perhaps a Metaverse sighting.\",\n", " \"snippets\": [\n", " \"Meta\\u2019s annual Connect conference started today, and this means lots of new hardware. Are you ready for an update on Meta Quest 3? Didn\\u2019t have time to tune in live? That\\u2019s okay \\u2014 we summed up the most important parts from the keynote below. And also what was missing. By that, of course, we mean the metaverse.\\nMeta Quest 3\\nConsider this a bonus treat for having to wait an extra 30 minutes for the keynote to begin. Meet Meta\\u2019s Quest 3, the headset model with improved passthrough tech, higher resolution displays and better graphics. There was also a tease for Meta Quest for business.\\n- Full-color passthrough tech has 10x as many pixels as its predecessor.\\n- 110-degree field of view.\\n- System is powered by the newly announced Qualcomm Snapdragon XR2 Gen 2 chip.\\n- Retails for $500.\\n- Comes with six months of the Quest+ VR subscription.\\n- Our U.S. managing editor Darrell Etherington tested out the Meta Quest 3 VR. Find out if he liked it.\",\n", " \"Also today, Meta launched AI Studio, a platform that\\u2019ll let businesses build AI chatbots for the company\\u2019s messaging services, including Facebook, Instagram and Messenger. Starting with Messenger, AI Studio will let companies \\u201ccreate AIs that reflect their brand\\u2019s values and improve customer service experiences,\\u201d according to the company. It is only available in alpha to start.\\nSpeaking of AI, get a load of the vast list of celebrities and influencers who have signed on to let Meta use their likeness for some fun. There are now 28 AI characters based on famous people \\u2014 but built entirely from AI \\u2014 from across the worlds of sport, music, social media and more.\",\n", " \"- Headspace, a \\u201cvirtual playground for your mind,\\u201d is coming.\\n- Learn about how \\u201caugments\\u201d can personalize your physical space.\\nXbox Cloud Gaming\\nWith the hardware news, Meta CEO Mark Zuckerberg announced the next generation of Meta Quest software. Roblox included.\\n- A virtual screen that can float in either a virtual or mixed-reality space.\\n- Appears to be reposition-able and resizable.\\n- Coming in December.\\nEmu\\nMeta\\u2019s new foundational model for image generation, Emu, comes with some fun things. One is generative AI stickers that are coming to Meta\\u2019s messaging apps. It will allow users to create unique AI stickers in a matter of seconds across Meta apps, including WhatsApp, Messenger, Instagram and even Facebook Stories.\\nRay-Ban Meta smart glasses\\nLooking for some new glasses? Meta has you covered with its new Ray-Ban smart glasses.\\n- Two round modules on the side of either eye include a 12-megapixel camera and an LED light that flips on to alert others that you\\u2019re recording.\",\n", " \"- You can livestream to friends and followers from the glasses.\\n- There are more than 150 design combos possible when you factor in frame color, style and lenses.\\n- You can preorder now in certain markets. The price starts at $299 for standard lenses. Polarized run $329 and transitions $379.\\n- Coming soon: the ability to translate text.\\nMeta AI\\nWe can tell Meta is angling for a seat at the AI chatbot table with the launch of a host of new AI-powered bots across its messaging apps. One is an AI assistant called Meta AI, which will soon come to Meta\\u2019s newly announced Quest 3 VR headset. It can help plan a trip with friends in a group chat, answer general-knowledge questions, and search the internet across Microsoft\\u2019s Bing to provide real-time web results.\"\n", " ],\n", " \"title\": \"Meta Connect 2023: Everything you need to know about Quest 3 VR, ...\",\n", " \"url\": \"https://techcrunch.com/2023/09/27/meta-connect-2023-quest-3-vr-everything-you-need-to-know/\"\n", " },\n", " {\n", " \"description\": \"We\\u2019re also building a sandbox ... to grow and evolve, we\\u2019ll bring this sandbox to the metaverse, giving you the chance to build AIs that adopt an even greater level of realism, embodiment, and connectedness....\",\n", " \"snippets\": [\n", " \"Here\\u2019s an example of how you might use Meta AI:\\nImagine you and your friends are in a group chat discussing which trailhead to try in Santa Cruz. Meta AI surfaces options directly in the chat, so you can decide as a group which location to explore. What if after the hike you want a creative way to commemorate the day? Meta AI can help. Type \\u201c@MetaAI /imagine\\u201d followed by a descriptive text prompt like \\u201ccreate a button badge with a hiker and redwood trees,\\u201d and it will create a digital merit badge in the chat with your friends.\\nA Universe of Characters at Your Fingertips\\nOur journey with AIs is just beginning, and it isn\\u2019t purely about building AIs that only answer questions. We\\u2019ve been creating AIs that have more personality, opinions, and interests, and are a bit more fun to interact with. Along with Meta AI, there are 28 more AIs that you can message on WhatsApp, Messenger, and Instagram. You can think of these AIs as a new cast of characters \\u2013 all with unique backstories.\",\n", " \"We know how important transparency is when it comes to the content AI generates, so images created with restyle and backdrop will indicate the use of AI to reduce the chances of people mistaking them for human-generated content. We\\u2019re also experimenting with forms of visible and invisible markers.\\nWe want these experiences to be safe and trustworthy, while bringing new forms of creativity, entertainment, and expression into your day.\\nAn Assistant That Spans Our Apps and Devices\\nMeta AI is a new assistant you can interact with like a person, available on WhatsApp, Messenger, Instagram, and coming soon to Ray-Ban Meta smart glasses and Quest 3. It\\u2019s powered by a custom model that leverages technology from Llama 2 and our latest large language model (LLM) research. In text-based chats, Meta AI has access to real-time information through our search partnership with Bing and offers a tool for image generation.\\nHere\\u2019s an example of how you might use Meta AI:\",\n", " \"AI is enabling new forms of connection and expression, thanks to the power of generative technologies. And today at Connect, we introduced you to new AI experiences and features that can enhance your connections with others \\u2013 and give you the tools to be more creative, expressive, and productive.\\nAI Stickers\",\n", " \"We are committed to building responsibly with safety in mind. We are continuing to test and evolve the capabilities of our AIs, and will improve the experience over time through what we learn from your interactions with them. Your direct feedback and the conversations you have with our AIs are core parts of what will help us improve our AI models, and ultimately enhance the experience at scale.\\nWhat\\u2019s Coming Next\\nWe introduced AI studio today, the platform that supports the creation of our AIs and we plan to make it available for people outside of Meta \\u2013 coders and non-coders alike \\u2013 to build AIs. Developers will be able to build third-party AIs for our messaging services with our APIs in the coming weeks, starting on Messenger then expanding to WhatsApp.\",\n", " \"- Roy Choi as Max, Seasoned sous chef for culinary tips and tricks\\n- Sam Kerr as Sally, Free-spirited friend who\\u2019ll tell you when to take a deep breath\\n- Snoop Dogg as Dungeon Master, Choose your own adventure with the Dungeon Master\\n- Tom Brady as Bru, Wisecracking sports debater who pulls no punches\\nWe\\u2019re going to start rolling these out in beta in the United States today. We\\u2019ll add new characters in the coming weeks played by Bear Grylls, Chloe Kim, and Josh Richards among others.\\nIt\\u2019s still early days for our AIs. Right now, their knowledge base \\u2013 with the exception of Meta AI, Bru, and Perry \\u2013 is limited to information that largely existed prior to 2023, which means some responses may be dated. We aim to bring search to many more of our AIs in the coming months \\u2013 like we have done with Meta AI \\u2013 so that conversations can be timely and relevant too.\",\n", " \"Businesses will also be able to create AIs that reflect their brand\\u2019s values and improve customer service experiences. From small businesses looking to scale to large brands wanting to enhance communications, AIs can help businesses engage with their customers across our apps. We\\u2019re launching this in alpha and will scale it further next year.\\nAnd for creators, they\\u2019ll be able to build AIs that extend their virtual presence across our apps. These AIs will have to be sanctioned by them and directly controlled by the creator.\\nWe\\u2019re also building a sandbox that will be released in the coming year, enabling anyone to experiment with creating their own AI. As our universe of AIs continues to grow and evolve, we\\u2019ll bring this sandbox to the metaverse, giving you the chance to build AIs that adopt an even greater level of realism, embodiment, and connectedness.\",\n", " \"AI Stickers\\nBillions of stickers are sent across our platforms every month, adding another fun and creative way for people to communicate and express themselves. Today, we announced new AI stickers that enable you to effortlessly generate customized stickers for your chats and stories. Using technology from Llama 2 and our foundational model for image generation called Emu, our AI tool turns your text prompts into multiple unique, high-quality stickers in seconds. This new feature, which is rolling out to select English-language users over the next month in WhatsApp, Messenger, Instagram, and Facebook Stories, provides infinitely more options to convey how you\\u2019re feeling at any moment. AI stickers will roll out to select English language users over the next month.\\nImage Editing With AI\",\n", " \"Image Editing With AI\\nSoon, you\\u2019ll be able to transform your images or even co-create AI-generated images with friends. Restyle and backdrop \\u2013 two new features that are coming soon to Instagram \\u2013 use the technology from Emu. Backdrop also leverages learnings from our Segment Anything Model.\\nRestyle lets you reimagine your images by applying the visual styles you describe. Think of typing a descriptor like \\u201cwatercolor\\u201d or a more detailed prompt like \\u201ccollage from magazines and newspapers, torn edges\\u201d to describe the new look and feel of the image you want to create.\\nBackdrop changes the scene or background of your image. Prompts like \\u201cput me in front of a sublime aurora borealis\\u201d or \\u201csurrounded by puppies\\u201d will cue the tool to create an image of the primary subject in the foreground with the background you described.\",\n", " \"And because interacting with them should feel like talking to familiar people, we did something to build on this even further. We partnered with cultural icons and influencers to play and embody some of these AIs. They\\u2019ll each have profiles on Instagram and Facebook, so you can explore what they\\u2019re all about.\\n- Charli D\\u2019Amelio as Coco, Dance enthusiast\\n- Chris Paul as Perry, Pro golfer helping you perfect your stroke\\n- Dwyane Wade as Victor, Ironman triathlete motivating you to be your best self\\n- Izzy Adesanya as Luiz, Showy MMA prospect who can back up his trash talk\\n- Kendall Jenner as Billie, No-BS, ride-or-die companion\\n- LaurDIY as Dylan, Quirky DIY and Craft expert and companion for Gen Z\\n- MrBeast as Zach, The big brother who will roast you \\u2014 because he cares\\n- Naomi Osaka as Tamika, Anime-obsessed Sailor Senshi in training\\n- Paris Hilton as Amber, Detective partner for solving whodunnits\\n- Raven Ross as Angie, Workout class queen who balances fitness with meditation\"\n", " ],\n", " \"title\": \"Introducing New AI Experiences Across Our Family of Apps and Devices ...\",\n", " \"url\": \"https://about.fb.com/news/2023/09/introducing-ai-powered-assistants-characters-and-creative-tools/\"\n", " },\n", " {\n", " \"description\": \"Zuckerberg spoke at the Meta Connect conference, the social media company's biggest event of the year as well as its first in-person conference since the start of the pandemic. He also said the latest Quest mixed-reality headset would start shipping on Oct. 10 and introduced the company's first consumer-facing generative AI products. The latter includes a chatbot called Meta ...\",\n", " \"snippets\": [\n", " \"The device will incorporate a new Meta AI assistant and be capable of livestreaming broadcasts of what a user is seeing directly to Facebook and Instagram, an advancement over the previous generation's ability to snap photos.\\nZuckerberg spoke at the Meta Connect conference, the social media company's biggest event of the year as well as its first in-person conference since the start of the pandemic.\\nHe also said the latest Quest mixed-reality headset would start shipping on Oct. 10 and introduced the company's first consumer-facing generative AI products. The latter includes a chatbot called Meta AI that can generate both text responses and photo-realistic images.\\n\\\"Sometimes we innovate by releasing something that's never been seen before,\\\" Zuckerberg said. \\\"But sometimes we innovate by taking something that is awesome, but super expensive, and making it so it can be affordable for everyone or even free.\\\"\",\n", " \"\\\"We've tried to exclude datasets that have a heavy preponderance of personal information,\\\" Clegg said, citing LinkedIn as an example of a website whose content was deliberately not used.\\nCUSTOM AI BOTS\\nMeta also announced that it was building a platform that developers and ordinary people alike may use to create custom AI bots of their own, which will have profiles on Instagram and Facebook and eventually appear as avatars in the metaverse.\\nTo demonstrate the tool's capabilities, Meta created a set of 28 chatbots with different personalities styled in the voices of celebrities like Charli D'Amelio, Snoop Dogg and Tom Brady, according to a company blog post.\\nThe features appeared to be aimed at sprucing up existing apps and devices rather than developing new ad surfaces or other sources of revenue.\",\n", " \"Meta unveils AI assistant, Facebook-streaming glasses\\nMENLO PARK, California, Sept 27 (Reuters) - Meta Platforms (META.O) Chief Executive Mark Zuckerberg on Wednesday rolled out new AI products for consumers, including bots that create photo-realistic images and smart glasses that answer questions, as well as an updated virtual-reality headset.\\nZuckerberg described the products as bringing together virtual and real worlds, and underscored that part of what Meta offered was low cost or free AI that could integrate into daily routine. Meta's Quest is the bestseller in the nascent VR space and the company's executives described it as the best value in the industry, a nod to the impending release of a much more expensive headset from Apple (AAPL.O).\\nSpeaking from a central courtyard on Meta's sprawling Silicon Valley campus, Zuckerberg said a new generation of Meta's Ray-Ban (ESLX.PA) smart glasses would start shipping on Oct. 17, priced at $299.\",\n", " \"Meta AI will be built in to the smart glasses as an assistant, starting with a beta rollout in the United States. A software update planned for next year will give the assistant the ability to identify places and objects that people are seeing, as well as to perform language translation.\\nMeta made Meta AI using a custom model based on the powerful Llama 2 large language model that the company released for public commercial use in July. The chatbot will have access to real-time information via a partnership with Microsoft's (MSFT.O) Bing search engine, Zuckerberg said.\\nIn an interview with Reuters, Meta Global Affairs President Nick Clegg said the company had taken steps to filter private details from the data used to train the model and also imposed restrictions on what the tool could generate, like a ban on the creation of realistic images of public figures.\",\n", " \"\\\"I don't see monetization of AI products happening for Meta for quite some time and I think it will end up being more indirect. They seem much more interested in helping develop a platform that other developers will use,\\\" said Bob O'Donnell, chief analyst at TECHnalysis Research.\\nZuckerberg also said on Wednesday that Xbox cloud gaming is coming to Quest in December.\\nMeta first announced the Quest 3 headset over the summer, around the time Apple debuted its Vision Pro headset, a high-end product with a price of $3,500.\\nStarting at $500, the Quest 3 boasts the same mixed-reality technology that premiered in Meta's more expensive Quest Pro device launched last year, which shows wearers a video feed of the real world around them.\\nThe day's announcements reflect how Zuckerberg plans to navigate the shift this year of investor fervor to artificial intelligence from augmented and virtual reality technologies.\",\n", " \"Stakes for the event were high as investors last year slammed the parent company of Facebook and Instagram for spending extensively on the metaverse, prompting Zuckerberg to lay off tens of thousands of staff to continue funding his vision.\\nDevelopers were watching to assess what apps they might create for Meta's latest hardware devices. Investors, meanwhile, looked for signs of whether a gamble that has lost the company more than $40 billion since 2021 may pay off.\\nReporting by Katie Paul and Anna Tong in Menlo Park, California Additional reporting by Yuvraj Malik, Pushkala Aripaka and Shashwat Awashti in Bengaluru Editing by Kenneth Li, Peter Henderson and Matthew Lewis\\nOur Standards: The Thomson Reuters Trust Principles.\"\n", " ],\n", " \"title\": \"Meta unveils AI assistant, Facebook-streaming glasses | Reuters\",\n", " \"url\": \"https://www.reuters.com/technology/meta-signal-future-arvr-investments-annual-connect-conference-2023-09-27/\"\n", " },\n", " {\n", " \"description\": \"NFTs are the new way to connect with others around the world. Meta Connect is the first true social hub focused toward NFT enthusiasts that truly enables its users. This platform will provide new ways to make friends, build your network, organize events, and promote your collection.\",\n", " \"snippets\": [\n", " \"What is Meta Connect?\\nNFTs are the new social. NFTs are the new way to connect with others around the world. Meta Connect is the first true social hub focused toward NFT enthusiasts that truly enables its users. This platform will provide new ways to make friends, build your network, organize events, and promote your collection. Enhance the value of your brand with the game-changing platform Meta Connect.\\nEnable location to appear on the map. Connect with other users and search for places to explore. Enable Ghost Mode when you want to go unnoticed.\\nSynchronize your wallet, access your blue-chip NFTs, and choose one to be your avatar.\\nPartnering with Meta Connect will provide your brand or business prioritized exposure to our valued user base. Want to register your brand or business with us? Fill out the contact form below!\",\n", " \"Stay current with the public feed. Keep in touch with friends and spark new conversations with nearby users. Share experiences or stories with picture and vocal messages.\\nJOIN US\\n2022 META CONNECT. ALL RIGHTS RESERVED\"\n", " ],\n", " \"title\": \"Meta Connect\",\n", " \"url\": \"https://meta-connect.io/\"\n", " },\n", " {\n", " \"description\": \"Meta Connect is Meta's annual hardware showcase, and this year the star of the show is the Meta Quest 3. But the standalone VR headset wasn't alone.\",\n", " \"snippets\": [\n", " \"Meta Connect is Meta's annual hardware showcase, and this year the star of the show is the Meta Quest 3. But the standalone VR headset wasn't alone.\\nMeta announced a bunch of new products, of course including the Meta Quest 3, but Emu AI and the new Rayban Meta Smartglasses were also there, apparently moving toward Meta's stated goal of furthering human connection \\u2013 whatever that means. We went ahead and gathered everything announced at the show right here, so you can find out what matters to you.\\nMeta Quest 3 - Mixed Reality for All?\\nThe Meta Quest 2 was a virtual reality headset, and while the Meta Quest 3 can do that, it differs by having cameras that allow your content to exist in the same world you do. It does this with two dedicated color sensors and a depth sensor that will allow you to do things like previewing furniture in your room.\",\n", " \"Of course, Meta is also using this as an opportunity to create Meta AI, an AI chatbot where you can ask questions and get AI-generated answers. But instead of it being one single chatbot, it's a wide collection of different chatbots that each specialize in their own special categories, and of course they're \\\"played\\\" by celebrities.\\nRayban Meta Smartglasses - Stylish Streaming?\\nThe Rayban Meta Smartglasses are back, and this time Meta offers even more styles. These are also the first smartglasses that are shipping with Meta AI built-in. You can use the glasses to interact with AI whenever you want.\",\n", " \"Emu - AI Comes to Instagram\\nAI has been everywhere over the last year, but Meta is looking to democratize the tech further. Emu, then, isn't really a product, but more an initiative to bring AI models to Facebook and Instagram. The idea here is to use AI image generation and manipulation to create fun photos to send to friends and even create custom stickers at the drop of a dime, whether you're in chat or commenting on a friend's post.\\nBeyond Meta's own AI models, the company is using this to create a platform for developers to develop AI models for Meta's massive userbase. So, get ready for a deluge of \\\"smart\\\" video filters - an even bigger deluge than we already have.\",\n", " \"Meta Quest 3 will also support Xbox Cloud Streaming. That doesn't mean that everything on Xbox Game Pass will be VR, but instead it's taking the Apple Vision Pro approach, projecting a giant virtual screen into your environment that you can sit and play games on with a regular controller.\\nUnder the hood, the Meta Quest 3 is powered by the Snapdragon XR2 Gen 2, which has a massively more powerful GPU and will allow more VR games to run natively on the Quest, when previously you'd have to play games tethered to one of the best gaming PCs.\\nMeta claims the Meta Quest 3 is the most powerful VR and MR headset ever, but we'll have to wait and see how it pans out in our review. However, with the price tag going up to $499, it might be a little harder to afford for \\\"everyone.\\\" But either way, you can buy it Oct. 10, 2023.\\nEmu - AI Comes to Instagram\",\n", " \"You can use the AI to ask questions and get real-time answers projected into the glasses themselves. And starting next year, this AI can do everything from translating signs you spot out in the wild to helping you solve problems that show up in the wild. It can do this by using built-in cameras to scan what you're looking at, so if we're going off of Meta's example, you can look at a leaky faucet, ask Meta AI how to fix it, and get advice.\\nYou'll also be able to livestream your point of view from your Rayban Meta Smartglasses to your audience, whether that's just friends and family or a wider group of people.\\nIf you want these new smartglasses, you can get them Oct. 17, starting at $299.\"\n", " ],\n", " \"title\": \"Meta Connect 2023: Everything Announced at the Meta Quest 3 Event ...\",\n", " \"url\": \"https://www.ign.com/articles/meta-connect-2023-everything-announced-at-the-meta-quest-3-event\"\n", " },\n", " {\n", " \"description\": \"At the Meta Connect 3 event, the company will launch its latest mixed-reality headset . Here's where you can watch the event live.\",\n", " \"snippets\": [\n", " \"How to watch Meta Connect 2023: Meta Quest 3 & more\\nMeta Connect 2023\\nMeta is all set to host Meta Connect 2023. At this annual event, the company will launch its latest mixed-reality headset, the Meta Quest 3. Here\\u2019s where you can watch the event live.\\nMeta Quest 3 is one of the most anticipated upcoming mixed reality headsets. Since it was first teased by Meta\\u2019s CEO, Mark Zuckerberg, earlier this year, experts have been talking highly about the new hardware.\\nHowever, the company only revealed a few details and announced that the Quest 3 will be launched later this year, at the company\\u2019s annual Meta Connect event.\\nFor the uninitiated, Meta Connect is a two-day event where the company will showcase its upcoming products, including the Quest 3, focusing on AI and virtual, mixed, and augmented realities.\",\n", " \"The highlight of this event will be the keynote, where Meta is expected to announce the Quest 3 mixed-reality headset. The company will also likely spend more time discussing the Quest 3\\u2019s features, pricing, and availability.\\nWhere can I watch the Meta Connect 2023 livestream?\\nThe Meta Connect 2023 virtual event will start on September 27 at 10 AM PST / 1 PM EST / 6 PM BST. The company will stream the event live on its website. You can also watch the stream on YouTube, Twitch, and its official page on Facebook.\\nThe event will start with Mark Zuckerberg\\u2019s keynote, including the Quest 3 launch and how Meta plans to bring mixed reality to life.\\nMeta Connect 2023: What to expect\\nThe highlight of Meta Connect 2023 will be the new Quest 3 mixed-reality headset, which will succeed the affordable Quest 2 VR headset.\",\n", " \"We already have some details about the Quest 3 from the earlier announcement. It will retail at $499, and while we expect it to be available in multiple storage variants, so $499 could be the price of the base variant. The Quest 3 will be more powerful and compact than the Quest 2.\\nIt will house a pancake lens and be powered by a Snapdragon XR2 Gen 2 processor. The Quest 3 will also come with a pair of new controllers and will sport a display with higher resolution than the Quest 2.\\nBest the mixed-reality headset, the company will talk about its developer ecosystem and offer a sneak peek into the Mixed-reality platform it is building.\\nMeta might also introduce Meta Quest+ at this year\\u2019s Connect event. A $7.99 subscription service similar to Xbox Game Pass, targeting new/first-time users.\\nBesides all the AR, VR, and MR talks, the company will also discuss the Llama AI ecosystem, too.\"\n", " ],\n", " \"title\": \"How to watch Meta Connect 2023: Meta Quest 3 & more - Dexerto\",\n", " \"url\": \"https://www.dexerto.com/tech/how-to-watch-meta-connect-2023-2311523/\"\n", " },\n", " {\n", " \"description\": \"After months of anticipation, we finally had a chance to strap on Meta's Quest 3 VR headset, and it's a game-changing device.\",\n", " \"snippets\": [\n", " \"How to watch Meta Connect 2023\\nYou can watch the entire Meta Connect 2023 event either on Facebook/Meta or through the Horizon\\u2019s World app on your Quest or Quest 2 headset. You\\u2019ll need a Facebook account for the latter, so sign up now if you haven\\u2019t already. Meta has also said it will stream it on \\u201cvarious social channels,\\u201d so you\\u2019ll probably be able to find a stream on its Twitter/X feed, and its YouTube channel.\\nAlternatively, you can register on the official website and attend the Meta headquarters to see the event live and in person. This is the first time since 2019 that Meta has had a live show like this.\\nThe event will kick off at 10 a.m. PT/1 p.m. ET on September 27, with Mark Zuckerberg\\u2019s keynote address.\\nWhat to expect from Meta Connect 2023\",\n", " \"Meta is also keen to push its software developments and will highlight some notable games and mixed reality experiences that it hopes will help push the Quest 3 to a new generation of VR and mixed reality users. It will also talk about artificial intelligence (AI) with Zuckerberg himself keen to highlight the ongoing developments of AI at Meta, and how it will help augment virtual reality experiences. Since OpenAI\\u2019s launch of ChatGPT, Meta has been working on its own large language model AIs and AI-driven applications, so may well showcase them at Connect 2023.\\nOther announced Meta Connect talks and presentations include a look at more realistic avatars, neural interfaces, and discussions for developers on how to improve their VR games and experiences.\\nEditors' Recommendations\\n- Meta Quest 3: everything you need to know\\n- How to use Meta Threads on the web\\n- This VR headset made me never want to go back to my Quest 2\\n- Apple Reality Pro vs. Meta Quest 3: power or price?\",\n", " \"What to expect from Meta Connect 2023\\nThe big announcement that will excite everyone, but cause little surprise, is the Meta Quest 3. This next-generation mainstream VR headset has been teased for quite some time, with Mark Zuckerberg also revealing tantalizing details about the VR device over the last few weeks, including that it would be notably thinner and lighter than its contemporaries, like the Quest Pro, and Apple\\u2019s planned Vision Pro mixed reality headset.\\nThe Quest 3 is expected to push mixed reality, too, with a focus on high-fidelity camera pass-through. It will have redesigned controllers, come with much faster onboard processing, and improved visuals through enhanced screen resolution. The price tag is slated to be $500, though we may see some optional versions with more storage and a higher price tag.\",\n", " \"Keen to get the hot insights on Meta\\u2019s next-generation Quest 3 headset? Us too. That\\u2019s while we\\u2019ll be tuned into the live stream for Meta Connect 2023 as soon as it goes live. If you want to join us, you\\u2019re more than welcome. It\\u2019s streaming live on Facebook, as you might expect, but if you want to really step into the future you can watch it in your Quest or Quest 2 headset in the the Horizon World app. You just might be a bit lonely. Active player counts are in the 10s.\\nThere will be lots of announcements and showcases over the two-day event, which runs from September 27 through 28, kicking off with the big keynote from Mark Zuckerberg on the 27th.\\nHow to watch Meta Connect 2023\",\n", " \"- The best VR headsets for 2023\"\n", " ],\n", " \"title\": \"How to watch Meta Connect 2023 today\",\n", " \"url\": \"https://www.digitaltrends.com/computing/how-watch-meta-connect-2023/\"\n", " },\n", " {\n", " \"description\": \"Meta Connect 3 is September 27, and we're going to see exactly what the Meta Quest 3 can do. Here's how you can watch the show.\",\n", " \"snippets\": [\n", " \"When is Meta Connect 3?\\nMeta Connect 3 takes place September 27, 2023, and will be spread across two days. The main keynote, however, will happen September 27 at 10 AM PST / 1PM EST / 5 PM BST. This will be a virtual event, with several breakouts happening after the main event. The biggest of these is the Developer State of the Union, which will be the place to go if you want to learn about developing for Meta's platforms.\\nWhere Can I Watch Meta Connect 3 Online?\\nIf you want to see what's next for the Meta Quest, the best place to watch Meta Connect 2023 is on Meta's own website. Meta will also likely stream the event on YouTube and Twitch, and we'll update this article as soon as we have links for those streams.\\nWhat to Expect From Meta Connect 3\",\n", " \"What to Expect From Meta Connect 3\\nWe already know that the Meta Quest 3 is going to be the star of the show at Meta Connect 3. While we do already know it's coming, there's very little we actually know about the upcoming headset. Back in June, Meta released a little announcement trailer showing off the next-generation VR headset, along with its $499 price tag.\\nAt Meta Connect 3, however, we'll likely see what's powering the standalone headset. All we know in this regard is that once again it will be powered by a Snapdragon SoC and apparently features twice the GPU power as the Meta Quest 3. Meta somehow found a way to do this while also slimming down the headset itself, so we'd expect Meta to go into some detail about how it made this happen. After all, with this kind of horsepower, thermals are a problem, and slimming down the device is going to take some pretty impressive engineering.\",\n", " \"We might already know the Meta Quest 3 is on the way, but we don't know what's actually going to packed inside the next best VR headset. Luckily, with Meta Connect starting tomorrow, that question, and many more are going to be answered.\\nMeta Connect 3 is going to be a two day event, likely packed with VR games and apps, but the main event is going to be the keynote, happening on September 27 at 10 AM PST / 1PM EST / 5 PM BST. Unlike a lot of tech events, the Meta Quest 3 is confirmed to be there. What this event is for, then, is showing off what the Meta Quest 3 can actually do. And with Meta hot off the heels of the Apple Vision Pro reveal, we expect it's going to be a huge show.\\nWhen is Meta Connect 3?\",\n", " \"We also know that along with the faster GPU, the Meta Quest 3 will have higher-resolution displays than the last go-around. Again, Meta has just claimed that the displays are a higher resolution, it hasn't said what the resolution would be. The Meta Quest 2 already boasted an impressive 1,920 x 1,832 resolution per eye, so Meta may have stepped up to a vertical resolution of 2,560, to match the boom in QHD monitors the best gaming PCs have been experiencing. However, that would require a heap more GPU power to pull off, so the \\\"faster GPU\\\" better be real.\\nBeyond that, we expect Meta to flesh out its Meta Quest+ a bit more to entice new users into its ecosystem. For those unaware, Meta Quest+ is a $7.99 a month subscription service that offers a wide variety of VR titles. Think of it as Xbox Game Pass, but for VR content.\"\n", " ],\n", " \"title\": \"Meta Connect 2023: How to Watch and What to Expect - IGN\",\n", " \"url\": \"https://www.ign.com/articles/meta-connect-2023-how-to-watch\"\n", " }\n", " ],\n", " \"latency\": 0.8664102554321289\n", "}\n" ] } ], "source": [ "# check the query result in JSON\n", "import json\n", "\n", "print(json.dumps(data, indent=2))" ] }, { "cell_type": "code", "execution_count": 7, "id": "7c40e73f-ca13-4f4a-a753-e613df3d389e", "metadata": {}, "outputs": [], "source": [ "# one way to load the JSON result with key info stored as \"snippets\"\n", "from llama_index import download_loader\n", "\n", "JsonDataReader = download_loader(\"JsonDataReader\")\n", "loader = JsonDataReader()\n", "documents = loader.load_data([hit[\"snippets\"] for hit in data[\"hits\"]])\n", "\n", "# another way to add the snippets in the query result to documents - this can be handy if you just need to add a list of text strings to doc:\n", "# from llama_index import Document\n", "# snippets = [snippet for hit in data[\"hits\"] for snippet in hit[\"snippets\"]]\n", "# documents = [Document(text=s) for s in snippets]" ] }, { "cell_type": "code", "execution_count": 8, "id": "a5de3080-2c4b-479c-baba-793b3bee36ed", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "model_name='sentence-transformers/all-mpnet-base-v2' embed_batch_size=10 callback_manager=\n" ] } ], "source": [ "# use HuggingFace embeddings \n", "from langchain.embeddings.huggingface import HuggingFaceEmbeddings\n", "from llama_index import LangchainEmbedding\n", "\n", "# the default embedding model is sentence-transformers/all-mpnet-base-v2, which provides a good balance between speed and performance\n", "# to change the default one, call HuggingFaceEmbeddings(model_name=). \n", "# for more info see https://huggingface.co/blog/mteb. \n", "embeddings = LangchainEmbedding(HuggingFaceEmbeddings())\n", "print(embeddings)\n", "\n", "# create a ServiceContext instance to use Llama2 and custom embeddings\n", "service_context = ServiceContext.from_defaults(llm=llm, chunk_size=800, chunk_overlap=20, embed_model=embeddings)\n", "\n", "# create vector store index from the documents created above\n", "index = VectorStoreIndex.from_documents(documents, service_context=service_context)\n", "\n", "# create query engine from the index\n", "query_engine = index.as_query_engine(streaming=True)" ] }, { "cell_type": "code", "execution_count": 9, "id": "de91a191-d0f2-498e-88dc-b2b43423e0e5", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " Sure! Here's a summary of the provided text:\n", "\n", "Meta Connect is an annual conference hosted by Meta where they showcase new hardware and technology. This year, they announced the Meta Quest 3, a standalone VR headset with improved passthrough tech, higher resolution displays, and better graphics. They also introduced the Rayban Meta Smartglasses, which are stylish streaming glasses with built-in AI models. Additionally, Meta launched AI Studio, a platform for businesses to create AI chatbots for their messaging services. The event also featured keynote speakers and developer sessions, and attendees could explore a virtual world inspired by Meta's Men" ] } ], "source": [ "# ask Llama2 a summary question about the search result\n", "response = query_engine.query(\"give me a summary\")\n", "response.print_response_stream()" ] }, { "cell_type": "code", "execution_count": 10, "id": "72814b20-06aa-4da8-b4dd-f0b0d74a2ea0", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " Based on the context information provided, the following products were announced:\n", "\n", "1. New Meta AI assistant\n", "2. Facebook-streaming glasses\n", "3. Next generation of Meta Quest software\n", "4. Ray-Ban smart glasses\n", "5. Meta AI bots for consumers\n", "6. Virtual screen that can float in either a virtual or mixed-reality space (coming in December)\n", "7. Generative AI stickers for Meta's messaging apps\n", "8. Xbox Cloud Gaming." ] } ], "source": [ "# more questions\n", "query_engine.query(\"what products were announced\").print_response_stream()" ] }, { "cell_type": "code", "execution_count": 11, "id": "a65bc037-a689-476d-b529-0059a27bc949", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " Sure! Based on the provided context information, here's what I found out about the Meta AI assistant:\n", "\n", "Meta has announced a new AI assistant called \"Meta AI\" that will soon come to its newly announced Quest 3 VR headset. This assistant can help plan trips with friends in a group chat, answer general-knowledge questions, and search the internet to provide real-time web results. It's powered by a custom model that leverages technology from Llama 2 and the company's latest large language model (LLM) research.\n", "\n", "In text-based chats, Meta AI has access to real-time information through the company's search partnership with Bing. The assistant is designed to be interactive like a person and will be available on WhatsApp, Messenger, Instagram, and Ray-Ban smart glasses.\n", "\n", "Additionally, there are 28 more AIs that users can message on these platforms, each with unique backstories. These AIs are part of a new universe of characters that Meta is introducing, which aim to bring new forms of creativity, entertainment, and expression into people's daily lives.\n", "\n", "Overall, the Meta AI assistant seems like an exciting development in the field of AI-powered personal assistants, offering a range of features and capabilities that could enhance user experiences across various Meta platforms." ] } ], "source": [ "query_engine.query(\"tell me more about Meta AI assistant\").print_response_stream()" ] }, { "cell_type": "code", "execution_count": 12, "id": "16a56542", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " Based on the provided context information, generative AI stickers refer to a new feature announced by Meta that enables users to generate customized stickers for their chats and stories using artificial intelligence technology from Llama 2 and the foundational model for image generation called Emu. These stickers use text prompts to create multiple unique, high-quality stickers in seconds, providing an infinitely more option to convey how you're feeling at any moment. The feature is currently rolling out to select English language users over the next month in WhatsApp, Messenger, Instagram, and Facebook Stories." ] } ], "source": [ "query_engine.query(\"what are Generative AI stickers\").print_response_stream()" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.18" } }, "nbformat": 4, "nbformat_minor": 5 }