Image: © Tada Images/Stock.adobe.com
Known as SAM, the AI model can identify specific objects in an image or video, even if it has not encountered those items in training.
Meta has revealed a new AI model that can detect distinct objects in an image even if it has never seen it before.
In a blogpost shared yesterday (5 April), Meta introduced the Segment Anything Model (SAM) which it claims can identify objects in an image or video fed to it, even when it has not encountered those items in training. This means that a user can select any item in a given visual by simply clicking it or even using a text prompt.
The Facebook owner also unveiled an accompanying segmentation dataset based on more than a billion segmentation masks collected through its data engine. One of the largest of its kind, the dataset is designed to train general-purpose object segmentation models from open world images.
“SAM has learned a general notion of what objects are, and it can generate masks for any object in any image or any video, even including objects and image types that it had not encountered during training,” Meta wrote in the blogpost.
This means that the model is general enough to cover a broad set of use cases and can be used “out of the box” on new image domains, from underwater photos to cell microscopy, without requiring additional training.
Meta hopes SAM can be incorporated as part of broader AI systems that can have a variety of uses including understanding both text and visual aspects of a webpage. It also has applications in AR/VR where it could enable selecting an object based on a user’s gaze and then ‘lifting’ it into 3D.
The model’s use can also be extended to content creators to improve applications such as extracting image regions for collages or video editing, or to researchers studying natural occurrences on Earth or in space by localising animals or objects to study and track in video.
“We believe the possibilities are broad, and we are excited by the many potential use cases we haven’t even imagined yet,” the company wrote.
Earlier this year, Meta released a new research-focused large language model called Llama, or Large Language Model Meta AI, to join the increasing competition in the AI space.
10 things you need to know direct to your inbox every weekday. Sign up for the Daily Brief, Silicon Republic’s digest of essential sci-tech news.
