Aviz Networks Overview and Networking 3.0 Unpacked

Event: Networking Field Day 34

Appearance: Introducing Aviz Networks at Networking Field Day 34

Company: Aviz Networks

Video Links:

Personnel: Ilona Gabinsky

In this introduction, learn how to unlock the potential of your network with our data-centric Networking 3.0 stack, designed for open, cloud, and AI-first networks. The Aviz Networking 3.0 stack redefines the landscape with a vendor-agnostic approach, supporting a wide range of ASICs, switches, clouds, and LLMs. At Aviz we enable open-source SONiC for NOS and open-source LLMs for generative AI. In this video, witness live demos of our Open Network Enterprise Suite (ONES) and Network Copilot, showcasing their transformative impact on network management and operations. Explore our suite of products, including the Fabric Test Automation Suite (FTAS) to ensure the quality of SONiC deployments, ONES for advanced network management, Open Packet Broker (OPB) for enhanced network visibility, and Network Copilot for smarter networks.


Forward Networks How Many Devices Do You Have – Network Lifecycle and Why You Need to Care

Event: Networking Field Day 34

Appearance: Forward Networks Presents at Networking Field Day 34

Company: Forward Networks

Video Links:

Personnel: Michael Wynston

Before Fiserv became a Forward Networks customer, the network team was faced with solving several enterprise network challenges: addressing CVE events, managing the perimeter, audits and compliance findings, and smoothly enabling automation. In this customer testimonial, Michael describes how Forward Enterprise uniquely delivered solutions for the known list of challenges, and also provided unexpected value by lowering the cost of data and validating network intent + network inventory.


The Power of A.I. in a Forward Networks Digital Twin

Event: Networking Field Day 34

Appearance: Forward Networks Presents at Networking Field Day 34

Company: Forward Networks

Video Links:

Personnel: Nikhil Handigol

Nikhil Handigol demonstrates how generative A.I. in unlocks network insights and shows the delegates how Forward Networks now seamlessly incorporates generative A.I. into its network digital twin software. Inside the most popular product feature, Network Query Engine, every user can perform natural language queries to access crucial network data in seconds.


Unlock Network Security and Reliability with a Forward Networks Digital Twin

Event: Networking Field Day 34

Appearance: Forward Networks Presents at Networking Field Day 34

Company: Forward Networks

Video Links:

Personnel: Mike Lossmann

In this product introduction, Mike Lossmann explains the secret sauce inside Forward Network’s flagship digital twin software, Forward Enterprise, and how network digital twin benefits more than just “networking people”. Forward Enterprise analyzes every possible network behavior, traces where every packet could ever go, and gives every user a queryable vendor-independent data model so NetOps, SecOps, CloudOps and even Compliance teams can get a mathematically-accurate view of the network.


Qlik AI Demo Showcase

Event: AI Field Day 4

Appearance: Qlik Presents at AI Field Day 4

Company: Qlik

Video Links:

Personnel: Sean Stauth

Global Director of AI and Machine Learning, Sean Stauth, demonstrates Qlik’s AI capabilities in market today. In this presentation, he showcase the strategic tenants of a trusted AI foundation, the turnkey benefits of AI-enhanced Solutions, and create custom solutions with self-service AI using real-world use cases with proven AI value: sales, operations, human resources, and IT/information security. Demonstrations include a mix of “day-in-the-life” data science/ML engineering development and end-user experiences with applications from both an analyst’s and executive’s points of view.

Stauth begins with a discussion of the strategic tenets of a trusted AI foundation, the benefits of AI-enhanced solutions, and the creation of custom solutions with self-service AI using real-world use cases in sales, operations, human resources, and IT/information security. His demonstrations cover a range of scenarios, including the use of Qlik’s supervised learning AutoML capability, which has been used to develop over 150,000 models.

Stauth demonstrates the “day-in-the-life” of data science and machine learning development, as well as end-user experiences with applications, providing perspectives from both analysts and executives. He shows how to use the AutoML platform to predict customer churn and build predictive applications with what-if scenarios. He also highlights the importance of explainability in AI models, which allows users to understand why predictions are made and ensures trust in the AI system.

Additionally, Stauth demonstrates the integration of generative AI with Qlik apps in operational use cases, the use of Qlik’s AI assistant with Microsoft Teams for collaboration, and key driver analysis built on top of the predictive AI engine. He emphasizes that the goal is not just to build models but to create apps and solutions that deliver real value to end users.

Stauth also addresses questions about the licensing and pricing of Qlik’s AI capabilities, the differentiation between various types of AI, and the importance of having guardrails and ethical considerations in place when using AI. The presentation concludes with a demonstration of generative AI integration using Amazon Bedrock, showcasing how natural language queries can be used to interact with data and derive insights.


Qlik AI-Enhanced and AI Roadmap Deep Dive

Event: AI Field Day 4

Appearance: Qlik Presents at AI Field Day 4

Company: Qlik

Video Links:

Personnel: Nick Magnuson, Ryan Welsh

In this presentation, VP of AI Strategy, Nick Magnuson, will share Qlik’s product direction for helping customers mitigate the risk, embrace the complexity, and scale the impact of AI by creating a trusted AI foundation, benefiting from AI-enhanced solutions, and creating custom solutions with self-service AI. Grounded in market and customer research, he shares how Qlik’s vision for helping customers integrate generative AI across their organizations. Joining Nick is Ryan Welsh, Field CTO and former CEO of Kyndi, an innovator in natural language processing, search, and generative AI, recently acquired by Qlik.

Magnuson outlines a three-pillar strategy for AI implementation: establishing a trusted AI foundation, infusing AI into their data integration and analytics products, and enabling customers to build custom AI solutions. He emphasizes the importance of data foundations, including data veracity, processing, observation, quality, integration, governance, and lineage. He also discusses the use of AI to make data foundation tasks more efficient, such as using natural language to generate SQL queries. He addresses questions about where the AI runs and clarifies that Qlik provides the pipeline for data movement, not a data warehouse, and mentions the acquisition of Talend for enhancing their portfolio.

The presentation includes a discussion about the use of AutoML, which has seen rapid adoption and has helped customers like ARH, a regional health system, save significant amounts of money by predicting patient no-shows and taking appropriate actions. Magnuson also touches upon the roadmap for AI, which includes advanced analytics and time series forecasting. He stresses the importance of feedback in shaping their AI strategy and notes the shift towards practical applications of generative AI in 2023.

Ryan Welsh, Field CTO and former CEO of Kyndi, speaks about the acquisition of Kyndi by Qlik and its implications. Kyndi specialized in natural language processing, search, and generative AI, and their technology aimed to provide trustworthy answers from unstructured text data. Welsh highlights the challenges of making large language models understand domain-specific content and explains how Kyndi’s technology addresses this issue.


Qlik AI Strategy Overview

Event: AI Field Day 4

Appearance: Qlik Presents at AI Field Day 4

Company: Qlik

Video Links:

Personnel: Mary Kern, Nick Magnuson

A recognized leader in analytics and data integration and quality, Qlik is trusted by 40,000 customers and 500 non- profits worldwide to help them use data, analytics and AI to create value by solving theirs and the world’s greatest challenges. In this presentation, VP of Analytics and AI Portfolio Marketing, Mary Kern, introduces Qlik’s strategy for helping organizations mitigate the risk, embrace the complexity, and scale the impact of AI.

Kern highlights Qlik’s history of innovation in visual analytics and its continued efforts in analytics, data integration, data quality, AI, and automation. She notes that Qlik supports over 40,000 organizations and 500 NGOs worldwide in leveraging data and AI to solve significant challenges. Kern discusses how Qlik took a step back to understand what organizations want from generative AI, finding that companies seek a partner to help them safely embrace AI, manage its complexity, and scale its positive impact. Qlik aids customers in building a trusted foundation for AI through broad data connectivity, intelligent data pipelines, data transformation engines, robust data catalogs, and lineage for traceability and trust.

Qlik offers turnkey AI solutions, enabling organizations to scale data-driven decisions across their teams through automated insights, natural language experiences, proactive alerts, and notifications. They also provide self-service AI solutions, including AutoML, data science, and generative AI offerings, under the umbrella of Qlik Stage. This set of solutions is designed to bring AI strategies to life while minimizing risk, handling complexity, and maximizing AI’s impact. Kern explains that Qlik has been an AI-enhanced analytics leader for years, using early LLMs like BERT to deliver prompt experiences in analytics. Qlik’s Insight Advisor acts as an AI assistant, helping developers auto-generate visualizations and dashboards, and enabling business users to ask questions and receive relevant answers in various formats and languages.

Qlik’s acquisition of predictive and AutoML capabilities allows customers to scale their data science investments by creating machine learning models to solve business problems. Kern cites examples of customers using Qlik’s AI capabilities, such as Airbus, Schneider Electric, and JB Hunt. During the Q&A, Kern and her colleague Nick elaborate on Qlik’s AutoML, which makes modeling more accessible to technical users. They discuss Qlik’s ability to handle both structured and unstructured data, with a focus on integrating data from diverse sources to find patterns that humans cannot interpret alone. They emphasize Qlik’s agnostic approach to data sources and its support for direct queries and in-memory computation to manage data effectively. The conversation also touches on Qlik’s use of open-source algorithms and standardized practices in AutoML. The company aims to simplify the process from experimentation to deployment, allowing users to easily integrate AI and machine learning into their existing Qlik dashboards and decision-making processes. Qlik’s platform facilitates the ingestion of data, transformation, and deployment of AI solutions without requiring heavy customer engagement or complex hardware setups.


Wherever There’s Data, There’s Possibility with Qlik

Event: AI Field Day 4

Appearance: Qlik Presents at AI Field Day 4

Company: Qlik

Video Links:

Personnel: Miranda Foster

Miranda Foster, VP of Corporate Communications, kicks off Qlik’s “Wherever There’s Data, There’s Possibility” session at AI Field Day with an overview of objectives, speakers, and presentations.

Foster welcomes attendees and outlines the agenda for the session. She highlights that AI is a critical component of Qlik’s company strategy, portfolio growth, and market approach. She mentions a recent acquisition of Kyndi, a company related to their AI portfolio.

Miranda then introduces the speakers for the session and also announces their user conference, Click Connect, scheduled to take place in Orlando from June 3rd to 5th.


Taming Unstructured Data Orchestration with Hammerspace

Event: AI Field Day 4

Appearance: Hammerspace Presents at AI Field Day 4

Company: Hammerspace

Video Links:

Personnel: Chad Smith, Floyd Christofferson

In this session, Floyd Christofferson and Chad Smith from Hammerspace will step through the key capabilities of Hammerspace Global Data Environment software, and how it automates unstructured data orchestration across multi-vendor, multi-site, and often multi-cloud storage environments. It will focus specifically on solutions to the problems of when data that is needed for AI pipelines is distributed across silos, sites, and clouds.

Christofferson and Smith discuss the capabilities of Hammerspace’s Global Data Environment software for automating unstructured data orchestration across various storage environments, including multi-vendor, multi-site, and multi-cloud infrastructures. They focus on how this can be particularly beneficial for AI workflows, where data is often distributed across different locations and silos.

Hammerspace’s solution involves separating file system metadata from the actual data, elevating it above the infrastructure layer into a global metadata control plane. This allows for a common view of files across different storage systems and locations, enabling transparent and automated data orchestration without disrupting user access or requiring data movement.

The software is Linux-based and includes two components: Anvil servers for metadata control and DSX nodes for I/O handling. It supports multi-protocol access, including NFS, parallel NFS, and S3, and allows for the setting of objective-based policies for data management, including protection, tiering, and geographical considerations.

Hammerspace can be installed on various platforms, including bare metal, cloud instances, and VMs, and it facilitates seamless integration of on-premises storage with cloud resources. This enables use cases like bursting AI workloads to the cloud, managing data across global sites, and optimizing compute resource costs by automating data movement to the most cost-effective locations.

Floyd provides examples of Hammerspace’s application in different industries, such as online gaming, rocket company Blue Origin, and a data center in London that saves costs by orchestrating render jobs to cheaper cloud regions.


Accelerating AI Pipelines with Hammerspace

Event: AI Field Day 4

Appearance: Hammerspace Presents at AI Field Day 4

Company: Hammerspace

Video Links:

Personnel: Chad Smith, Floyd Christofferson

In this session, Floyd Christofferson and Chad Smith from Hammerspace will look at solutions to achieve HPC-class performance to feed GPU-based AI pipelines while leveraging data in place on existing storage resources. This session will give real-world examples of how customers have adapted their existing infrastructure to accommodate the performance levels needed for AI and other high-performance workflows.

Christofferson and Smith discuss how Hammerspace can accelerate AI pipelines by addressing the challenges of managing and accessing unstructured data across various storage systems and locations. They introduce the concept of a global data environment that leverages a parallel global file system, allowing data to remain in place while providing high-performance access necessary for AI workloads. They begin by explaining the silo problem in AI pipelines, where unstructured data is spread across multiple storage types and locations, making it difficult to aggregate without moving it to a new repository. Hammerspace’s solution allows for the assimilation of file system metadata from existing storage, enabling a global view and access to data without physically moving it. This approach prevents copy sprawl, maintains data governance, and avoids additional capital and operational expenses.

The session highlights the introduction of a new product, Hammerspace Hyperscale NAS, which provides HPC-class parallel file system performance using standard protocols and networking, without requiring proprietary clients or altering existing infrastructure. This solution is said to be storage agnostic and can accelerate existing third-party storage, making it suitable for enterprises looking to incorporate AI workflows without significant upfront investment. The duo provides real-world examples, including a hyperscaler with a large AI training and inferencing environment, where Hammerspace’s technology enabled scalability without altering the existing infrastructure. Another example is a visual effects customer who achieved the required performance for rendering without changing their storage infrastructure.


Nature Fresh Farms – Maximizing Greenhouse Yield Using AI Powered by Intel

Event: AI Field Day 4

Appearance: Nature Fresh Farms Presents with Intel at AI Field Day 4

Company: Intel, Nature Fresh Farms

Video Links:

Personnel: Keith Bradley

Keith Bradley, VP of IT and Security at Nature Fresh Farms, presented on how the company uses AI to maximize greenhouse yield. Nature Fresh Farms is a 250-acre greenhouse operation that grows bell peppers, tomatoes, cucumbers, and strawberries in a controlled environment, focusing on increasing yield per square meter. They started 23 years ago with a 16-acre greenhouse and a data-forward approach, using computers to control the environment.

Over the years, they have developed around 32 AI models for different aspects of the farm, such as plant growth, pest control, and resource management. These models help the greenhouse transition from a reactive to a proactive state, optimizing conditions before weather changes affect the plants. The models rely on various sensors that collect data on light, CO2 levels, irrigation, and plant nutrition. The technology infrastructure includes a core data center where all the sensor data is aggregated and analyzed.

Nature Fresh Farms has seen a consistent increase in yield due to their AI models and has a closed-loop water system that recycles 99% of the water used. They also utilize natural gas to heat the greenhouses and capture CO2 emissions to stimulate plant growth.

Keith’s team consists of a small DevOps team that works closely with growers to collect and analyze data, which is then used to optimize the entire process from growing to shipping. The team is mostly in-house, with some contributions from vendors, especially from Europe. The company’s IT infrastructure is built on a hyper-converged system that allows them to scale easily and maintain high availability, which is critical as even a few hours of downtime can significantly impact the crops.

The presentation highlighted the real-world financial impact of AI in agriculture, moving beyond stereotypes of AI applications and demonstrating tangible benefits in optimizing greenhouse operations.


Deploy AI Everywhere From Edge to Data Center and Cloud with Intel

Event: AI Field Day 4

Appearance: Intel Presents at AI Field Day 4

Company: Intel

Video Links:

Personnel: Ronak Shah

In this session, Ronak Shah, AI Product Director at Intel’s Data Center and AI Group, reflects on the full day of presentations at AI Field Day 4. In addition to Intel, the day included representatives from Nature Fresh Farms, VMware by Broadcom, Google Cloud, and Kamiwaza. He emphasizes the prevalence of AI deployments, particularly inferencing on CPUs, and why many customers opt for this approach.

  • Nature Fresh Farms is highlighted as a case study for meeting critical requirements using CPUs without the need to transition to other hardware, thus simplifying infrastructure and managing costs.
  • VMware’s perspective is summarized with the phrase “use CPUs when you can and GPUs when you must,” demonstrating that CPUs are sufficient for many smaller inferencing tasks.
  • Google Cloud’s input supports the notion that the majority of inference work is done on CPU instances, especially among enterprise customers with real-time, smaller models.
  • Kamiwaza’s contribution underlines the challenges of deploying AI systems and the importance of partners that simplify the journey for customers. The ease of deployment with CPUs and the quick adaptation to new generations, such as the fifth-gen Xeon, are also discussed.

The conversation touches on the evolving AI landscape, the need for education on AI deployment, and the importance of software in maximizing hardware potential.


Kamiwaza Private AI Inference Mesh and Data Engine

Event: AI Field Day 4

Appearance: Kamiwaza Presents with Intel at AI Field Day 4

Company: Kamiwaza.AI

Video Links:

Personnel: Luke Norris, Matt Wallace

Luke Norris and Matt Wallace from Kamiwaza presented their mission to help enterprises achieve a trillion inferences a day, which they believe is key to the fifth industrial revolution. They discussed the massive scale at which they aim to operate, targeting Fortune 500 and Global 2000 companies, and addressing real-world use cases and problems. They explained the origin of the company name Kamiwaza, which means “superhuman” and reflects their goal to bring superhuman capabilities to enterprises.

The presenters focused on the importance of inferencing at scale, rather than model training, as they believe the latter is better left to very few experts employed by major tech companies. They emphasized that enterprises will use multiple foundational models and will need to manage these models effectively for various tasks. Kamiwaza aims to provide a full-stack generative AI solution that addresses the core problems of scale for an enterprise.

They introduced two key features of their solution: the Inference Mesh and the Distributed Data Engine. These features allow AI deployment anywhere, including on-premises, cloud, core, and edge, and work on a variety of hardware. They explained that the Inference Mesh and Distributed Data Engine work together to route inference requests efficiently, even when the data is in different locations. This hybrid approach is designed to enable massive scale data processing with LLMs (Large Language Models).


Developer and Operational Productivity in Google Cloud with Duet AI

Event: AI Field Day 4

Appearance: Google Cloud Presents Cloud Inferencing with Intel at AI Field Day 4

Company: Google Cloud

Video Links:

Personnel: Ameer Abbas

In this session, we’ll demonstrate how Duet AI enhances developer and operational productivity. We’ll explore how Google’s state of the art AI is applied to address real- world development and operations challenges. Topics include context-aware code completion, licensing compliance assistance, code explanation, test generation, operational troubleshooting and more. We’ll share customer successes and insights from within Google that inform continuous improvement of AI productivity tools.

Ameer Abbas, Senior Product Manager at Google Cloud, provides a demonstration of Duet AI and its application in enhancing developer and operational productivity. He explains how Google’s state-of-the-art AI is applied to real-world development and operations challenges, emphasizing its role in assisting with context-aware code completion, licensing compliance, code explanation, test generation, operational troubleshooting, and more.

Ameer highlights the division of Google’s AI solutions for consumers and enterprises, mentioning products like Gemini (formerly Bard), Maker Suite, Palm API, Workspace, and Vertex AI. Vertex AI is a platform for expert practitioners to build, extend, tune, and serve their own machine learning models, while Duet AI offers ready-to-consume solutions built on top of foundational models.

He discusses the importance of modern applications that are dynamic, scalable, performant, and intelligent, and how they contribute to business outcomes. Ameer references the DevOps Research and Assessment (DORA) community and its focus on key metrics like lead time for changes, deployment frequency, failure rate, and recovery time for incidents.

The presentation includes a live demo where Ameer uses Duet AI within the Google Cloud Console and an Integrated Development Environment (IDE) to perform various tasks such as generating API specs, creating a Python Flask app, and troubleshooting errors. He demonstrates how Duet AI can understand and generate code based on prompts, interact with existing code files, and provide explanations and suggestions for code improvements. Ameer also shows how Duet AI can assist with generating unit tests, documentation, and fixing errors, and he touches on its capability to learn from user interactions for future improvements.

The demo showcases how Duet AI can be integrated into different stages of the software development lifecycle and how it can be a valuable tool for developers and operators in the cloud environment. Ameer concludes by mentioning future features like the ability to have Duet AI perform actions on the user’s behalf and the incorporation of agents for proactive assistance.


Google Cloud AI Platforms and Infrastructure

Event: AI Field Day 4

Appearance: Google Cloud Presents Cloud Inferencing with Intel at AI Field Day 4

Company: Google Cloud

Video Links:

Personnel: Brandon Royal

In this session, we’ll explore how Vertex AI, Google Kubernetes Engine (GKE) and Google Cloud’s AI Infrastructure provide a robust platform for AI development, training and inference. We’ll discuss hardware choices for inference (CPUs, GPUs, TPUs), showcasing real-world examples. We’ll cover distributed training and inference with GPUs/TPUs and optimizing AI performance on GKE using tools like autoscaling and dynamic workload scheduling.

Brandon Royal, product manager at Google Cloud, discusses the use of Google Cloud’s AI infrastructure for deploying AI on Google’s infrastructure. The session focuses on how Google Cloud is applying AI to solve customer problems and the trends in AI, particularly the platform shift towards generative AI. Brandon discusses the AI infrastructure designed for generative AI, covering topics such as inference, serving, training, fine-tuning, and how these are applied in Google Cloud.

Brandon explains the evolution of AI models, particularly open models, and their importance for flexibility in deployment and optimization. He highlights that many AI startups and unicorns choose Google Cloud for their AI infrastructure and platforms. He also introduces Gemma, a new open model released by Google DeepMind, which is lightweight, state-of-the-art, and built on the same technology as Google’s Gemini model. Gemma is available with open weights on platforms like Hugging Face and Kaggle.

The session then shifts to a discussion about AI platforms and infrastructure, with a focus on Kubernetes and Google Kubernetes Engine (GKE) as the foundation for open models. Brandon emphasizes the importance of flexibility, performance, and efficiency in AI workloads and how Google provides a managed experience with GKE Autopilot.

He also touches on the hardware choices for inference, including CPUs, GPUs, and TPUs, and how Google Cloud offers the largest selection of AI accelerators in the market. Brandon shares customer stories, such as Palo Alto Networks’ use of CPUs for deep learning models in threat detection systems. He also discusses the deployment of models on GKE, including autoscaling and dynamic workload scheduling.

Finally, Brandon provides a live demo of deploying the Gemma model on GKE, showcasing how to use the model for generating responses and how it can be augmented with retrieval-augmented generation for more grounded responses. He also demonstrates the use of Gradio, a chat-based interface for interacting with models, and discusses the scaling and management of AI workloads on Google Cloud.


AI without GPUs: Using Intel AMX CPUs on VMware vSphere with Tanzu Kubernetes

Event: AI Field Day 4

Appearance: VMware by Broadcom Presents Private AI with Intel at AI Field Day 4

Company: VMware by Broadcom

Video Links:

Personnel: Earl Ruby

Looking to deploy AI models using your existing data center investments? VMware and Intel have collaborated to announce VMware Private AI with Intel. VMware Private AI with Intel will help enterprises build and deploy private and secure AI models running on VMware Cloud Foundation and boost AI performance by harnessing Intel’s AI software suite and 4th Generation Intel® Xeon® Scalable Processors with built-in accelerators. In this session we’ll explain how to set up Tanzu Kubernetes to run AI/ML workloads that utilize AMX CPUs.

Earl Ruby, R&D engineer at VMware by Broadcom, presented deployment of AI models without GPUs, focusing on the use of Intel AMX CPUs with Tanzu Kubernetes on vSphere. He discussed the benefits of AMX, an AI accelerator built into Intel’s Sapphire Rapids and Emerald Rapids Xeon CPUs, which can run AI workloads without separate GPU accelerators. vSphere 8 supports AMX, and many ML frameworks are already optimized for Intel CPUs.

He demonstrated video processing with OpenVINO on vSphere 8, showing real-time processing with high frame rates on a VM with limited resources and no GPUs. This demonstration highlighted the power of AMX and OpenVINO’s model compression, which reduces memory and compute requirements.

For deploying AMX-powered workloads on Kubernetes, Earl explained that Tanzu is VMware’s Kubernetes distribution optimized for vSphere, with lifecycle management tools, storage, networking, and high availability features. He detailed the requirements for making AMX work on vSphere, including using hardware with Sapphire Rapids or Emerald Rapids CPUs, running the Linux kernel 5.16 or later, and using hardware version 20 for virtualizing AMX instructions.

Earl provided a guide for setting up Tanzu to use AMX, including adding a content library with the correct Tanzu Kubernetes releases (TKRs) and creating a new VM class. He showed how to create a cluster definition file for Tanzu Kubernetes clusters that specifies the use of the HWE kernel TKR and the AMX VM class for worker nodes.

Finally, he presented performance results of the Llama 2 7 billion LLM inference running on a single fourth-gen Xeon CPU, demonstrating that it could deliver inference with an average latency under 100 milliseconds, which is suitable for chatbot response times.


AI without GPUs: Using Intel AMX CPUs on VMware vSphere for LLMs

Event: AI Field Day 4

Appearance: VMware by Broadcom Presents Private AI with Intel at AI Field Day 4

Company: VMware by Broadcom

Video Links:

Personnel: Earl Ruby

Looking to deploy AI models using your existing data center investments? VMware and Intel have collaborated to announce VMware Private AI with Intel. VMware Private AI with Intel will help enterprises build and deploy private and secure AI models running on VMware Cloud Foundation and boost AI performance by harnessing Intel’s AI software suite and 4th Generation Intel® Xeon® Scalable Processors with built-in accelerators. In this session we’ll explain the technology behind AMX CPUs and demonstrate LLMs running on AMX CPUs.

Earl Ruby, R&D Engineer at VMware by Broadcom, discusses leveraging AI without the need for GPUs, focusing on using CPUs for AI workloads. He talks about VMware’s collaboration with Intel on VMware Private AI with Intel, which enables enterprises to build and deploy private AI models on-premises using VMware Cloud Foundation and Intel’s AI software suite along with the 4th Generation Intel Xeon Scalable Processors with built-in accelerators.

Ruby highlights the benefits of Private AI, including data privacy, intellectual property protection, and the use of established security tools in a vSphere environment. He explains the technology behind Intel’s Advanced Matrix Extensions (AMX) CPUs and how they can accelerate AI/ML workloads without the need for separate GPU accelerators. AMX CPUs are integrated into the core of Intel’s Sapphire Rapids and Emerald Rapids servers, allowing for the execution of AI and non-AI workloads in a virtualized environment. Ruby demonstrates the performance of Large Language Models (LLMs) running on AMX CPUs compared to older CPUs without AMX, showing a significant improvement in speed and efficiency.

He also discusses the operational considerations when choosing between CPU and GPU for AI workloads, emphasizing that CPUs should be used when performance is sufficient and cost or power consumption are concerns, while GPUs should be used for high-performance needs, especially when low latency or frequent fine-tuning of large models is required.


Deploy AI Everywhere on Intel Xeon CPUs

Event: AI Field Day 4

Appearance: Intel Presents at AI Field Day 4

Company: Intel

Video Links:

Personnel: Ronak Shah

There’s a major AI hype cycle today, but what do businesses actually need? Today’s enterprises typically benefit from AI as a general-purpose, mixed workload instead of a purely dedicated one. Intel AI Product Director Ro Shah contextualizes the time and place for inferencing, nimble vs giant AI models, hardware and software options – all with TCO in mind. He leads into customer and partner examples to ground this in reality and avoid the FOMO.

Ro Shah, AI Product Director at Intel, discusses the deployment of AI, particularly focusing on inferencing, on Intel Xeon CPUs. He explains that while deep learning training often requires accelerators, deployment can be effectively handled by a mix of CPUs and accelerators. Shah emphasizes that CPUs are a good fit for mixed general-purpose and AI workloads, offering ease of deployment and total cost of ownership (TCO) benefits.

Shah describes a customer usage model where AI deployment bifurcates into two scenarios: large-scale dedicated AI cycles, which may require accelerators, and mixed workloads with general-purpose and AI cycles, where CPUs are advantageous. He provides a threshold for model size, suggesting CPUs for models with less than 20 billion parameters, and accelerators for anything larger. Using customer examples, Shah illustrates the advantages of deploying AI on CPUs for mixed workloads, such as video conferencing with added AI features like real-time transcription and speech translation. He also touches on the capabilities of Intel CPUs in client-side applications and the potential for on-premises deployment for enterprise customers.

Shah moves on to discuss generative AI and the use of large language models, noting that CPUs can meet latency requirements up to about 20 billion parameters. He shows performance data for specific models, highlighting the importance of next-token latency in determining whether a CPU or an accelerator is appropriate for a given task.

Regarding software, Shah stresses the importance of upstreaming optimizations to standard tools like PyTorch and TensorFlow, and mentions Intel-specific tools like OpenVINO and Intel Neural Compressor for performance improvements. He also covers the ease of transitioning between Xeon generations and how Intel’s broad ecosystem presence allows for AI deployment everywhere.


Insights into AI from Futurum Intelligence

Event: AI Field Day 4

Appearance: Insights into AI from Futurum Intelligence

Company: The Futurum Group

Video Links:

Personnel: Stephen Foskett

Stephen Foskett discusses the Futurum Group’s Intelligence platform, which is focused on workplace intelligence, customer experience, and AI. Foskett demonstrates the Intelligence platform, which is based on surveys of IT decision-makers and is updated every six months. The AI market data presented is collected by Keith Kirkpatrick and includes information about AI platform usage, vendor partnerships, and plans for changing or adding vendors. The data is global and includes a variety of industries, not just IT companies.

The platform allows users to access detailed data, including the actual survey questions and demographic information of respondents. It also shows key findings, such as the percentage of people using specific vendors for SaaS products and end-to-end AI solutions. Users can filter data by industry, region, and other criteria.

Foskett also previews upcoming data on AI chipsets and DevOps tools. He emphasizes the usefulness of the data for industry professionals, including product marketers, managers, and analysts, who need to make informed decisions based on market trends and competition.


Discussing the VAST Data Solution for AI

Event: AI Field Day 4

Appearance: VAST Data Presents at AI Field Day 4

Company: VAST Data

Video Links:

Personnel: John Mao, Keith Townsend, Neeloy Bhattacharyya

In this discussion, Keith Townsend interviewed John Mao and Neeloy Bhattacharyya from VAST Data. They discuss the company’s recent growth, including closing a funding round that values the company at $9.1 billion, due in part to significant sales and growth in the data storage market. To start, Keith asks about a recent report that VAST Data has achieved a 6% share of the data center flash storage market, which would be notable for an independent software data platform company.

The conversation shifts to VAST Data’s role in AI, noting that about 60% of their business is used for AI and high-performance computing (HPC) workloads. VAST Data has been involved in AI before it became a trending topic and has been working with large customers and AI service providers. The discussion then moves on to the unique aspects of the VAST platform that make it suitable for AI workloads. They talk about the company’s vision and strategy, which extends beyond traditional storage to include capabilities that address the entire AI data pipeline. VAST Data’s global namespace, which is right-consistent and allows for distributed inference and model serving, is a key feature that facilitates the AI pipeline by providing a common dataset for different locations to access and work from.

They also discuss VAST’s multi-protocol platform and its disaggregated shared-everything architecture, which allows for intelligent data movement based on actual workload needs rather than pre-staging data. Keith asks about how VAST helps with the data gravity problem and the challenges of moving compute closer to the data. Neeloy explains that VAST’s architecture, including its highly scalable metadata layer, allows for a better understanding of data access patterns and more intelligent pre-staging of data.

Finally, they touch upon VAST’s DataBase, which helps with the data preparation phase by assigning structure to unstructured data and accelerating ETL tools and query engines. This reduces the time necessary for data preparation, which is a significant part of the AI project lifecycle.


« Previous Page
Next Page »