Azure AI Overview: Microsoft Services & Solutions

In this post we are going to look at artificial intelligence (AI) the various services offered by Microsoft within the Azure platform. There will be following posts that go into more detail this post will focus on giving a high-level overview of what is currently on offered.

AI covers a range of software functions that enable applications to show human-like behavior and skills (Generative AI, Computer Vision, Speech, Natural Language Processing). Whilst AI has been around for many years the features have evolved and recent AI solutions are built using machine learning and generative AI modules.

Generative AI

Generative AI is a recent advancement within the field of AI, that enables applications and users to generate new content. These AI models are trained on large volumes of data and allows for the generation of images, videos and text. Depending on the size of the training data the resulting models are classed as either Large Language Models (LLMs) or Small Language Models (SLMs), the larger models allow for more generalized tasks, where as SLMs are more focused but cheaper to run.

Generative AI has many common uses including AI Agents, translation of text between languages, summarizing or creating new documents.

Computer Vision

Computer Vision is type of AI that specializes in image interpretation. Training on a large of number of images, it enables programs image classification and object detection. Image classification allows for analysis of unlabelled images after training. Object detection focuses on locating objects within images.

These systems could be used for visual searching, facial recognition, tagging of photographs and one of the underlying technologies of self-driving cars.

Speech

Within AI there are a number of speech capabilities that can be used to interpret or create audio content. Speech recognition is used to convert speech or audio content to text, used for automated transcription of calls, meetings or describing the content of a video. Speech synthesis is the creation of audio files from text, or text-to-speech used as the basis of personal AI assistants found on phones or household devices for example Siri, Alexa.

Natural Language Processing

Natural Language Processing (NLP) can be used to analyze documents for sentiment, opinion or to identify the mentions of specific people or places. Some of these tasks are handled by generative AI models but simpler NLP models can also perform the tasks at more cost-effective ways. Including the creation of simple chatbots that don’t require the complexity of AI responses.

OCR and Data extraction

Optical Character Recognition (OCR) can be used to extract data from standard forms, using OCR to identify the location of text in images, with more advanced models interpreting the values with documents.

Azure AI Services

There are a range of Cloud Services that Azure offer to assist in your development of AI applications. One of these sets of services are Azure AI Services which are a set of prebuilt services and models that can be added to applications. These services include :

Azure OpenAI
The Azure OpenAI service provides access to OpenAI generative AI models including the GPT family of large and small language models and DALL-E image-generation models within a scalable and securable cloud service on Azure.

Vision
The Azure AI Vision service provides a set of models and APIs that you can use to implement common computer vision functionality in an application. With the AI Vision service, you can detect common objects in images, generate captions, descriptions, and tags based on image contents, and read text in images.

Face
The Azure AI Face service is a specialist computer vision implementation that can detect, analyze, and recognize human faces. Because of the potential risks associated with personal identification and misuse of this capability, access to some features of the AI Face service are restricted to approved customers.

Language
Azure AI Content Safety provides developers with access to advanced algorithms for processing images and text and flagging content that is potentially offensive, risky, or otherwise undesirable.

Speech
The Azure AI Speech service provides APIs that you can use to implement text to speech and speech to text transformation, as well as specialized speech-based capabilities like speaker recognition and translation.

Azure AI Search
The Azure AI Search service uses a pipeline of AI skills based on other Azure AI Services and custom code to extract information from content and create a searchable index. AI Search is commonly used to create vector indexes for data that can then be used to ground prompts submitted to generative AI language models, such as those provided in the Azure OpenAI service.

A full list of services offered are available within the MS Learn documentation here.

Azure AI services can be provisioned in the Azure Portal, as well as using Infrastructure as Code. Thes services can be deployed as single standalone resources, or as a multi-service AI service grouping a set of APIs into a single Azure resource, for example Azure AI Vision

Azure AI Foundry

Alongside the Azure AI services, Microsoft also supplies the development platform Azure AI Foundry. Microsoft recommends using AI Foundry for organizing and managing the AI resources for service consumption. Foundry offers features for management of your projects. Hubs are containers that can be used to manage shared configuration and data used with application development. Projects are a more specific container, used to collaborate on a single project.

Responsible AI Usage

Responsible AI involves ensuring fairness, reliability, safety, privacy, and inclusiveness in AI systems. Developers need to mitigate bias, protect personal data, be transparent about AI limitations, and apply governance frameworks to uphold these principles. By doing so, they can minimize discrimination, safeguard security, and ensure AI benefits are accessible to all.

Examples of responsible AI practices include testing AI models for fairness in college admissions, ensuring robotic systems act safely, protecting personal data in facial recognition, providing accessibility features for speech-based chatbots, and transparently disclosing AI use in loan approvals.

Fairness

AI systems must ensure fairness, avoiding biases like gender or ethnicity, through careful planning, representative data, and performance checks during development.

Reliability and safety

AI systems must ensure reliability and safety, especially in critical applications like autonomous vehicles or medical diagnostics. Rigorous testing, deployment management, and careful evaluation of confidence scores are essential to minimize risks

Privacy and security

AI systems must prioritize security and privacy, protecting personal data with safeguards during both training and production to address privacy and security concerns.

Inclusiveness

AI systems should empower and benefit all of society, promoting inclusiveness by involving diverse perspectives in design, development, and testing.

Transparency

AI systems should be transparent, ensuring users understand their purpose, functionality, and limitations. Key factors like training data, prediction accuracy, confidence scores, and data usage should be clearly communicated.

Accountability

Developers are accountable for AI systems, ensuring they meet responsibility requirements by training models, defining decision logic, and adhering to governance frameworks that uphold legal and ethical standards.

 Summary

In this blog post, we explored the various areas of artificial intelligence (AI) and the services offered by Microsoft within the Azure platform. We delved into the advancements in Generative AI, Computer Vision, Speech, Natural Language Processing, and Optical Character Recognition (OCR), highlighting their applications and benefits. Additionally, we discussed the range of Azure AI Services, including Azure OpenAI, Vision, Face, Language, Speech, and AI Search, which provide powerful tools for developers to create innovative AI solutions. Finally, looked at the importance of responsible AI usage, ensuring fairness, reliability, safety, privacy, inclusiveness, transparency, and accountability in AI systems to benefit all of society.