Meta launched a brand new synthetic intelligence (AI) mannequin on Monday that may carry out complicated laptop imaginative and prescient duties. Dubbed Phase Something Mannequin 2 (SAM 2), it follows after its predecessor that was launched final yr and was included in Instagram’s Backdrop and Cutouts instruments. The successor to the mannequin now comes with superior capabilities and the corporate stated it may carry out phase identification and monitoring even on movies. Like most of Meta’s giant language fashions (LLMs), SAM 2 can be an open-source AI mannequin.
In a newsroom publish, Meta introduced the brand new AI mannequin which focuses on phase evaluation on movies primarily, whereas bettering its picture segmentation capabilities. Highlighting the accomplishments of its predecessor, Meta stated the AI mannequin was utilized in Instagram’s Backdrop and Cutouts options, whereas marine scientists used it to “phase sonar pictures and analyse coral reefs, satellite tv for pc imagery evaluation for catastrophe reduction, and within the medical area, segmenting mobile pictures and aiding in detecting pores and skin most cancers”.
SAM 2 is able to object segmentation in a picture and video in addition to observe it throughout completely different frames of a video in real-time. The AI may observe and phase objects in eventualities the place the objects transfer quick, change in look, or are hid by different objects or a wholly completely different scene.
The inspiration mannequin for prompt-based visible segmentation is constructed on a easy transformer structure. It has a streaming reminiscence that enables it to course of movies in real-time. The corporate additionally claimed that the mannequin was educated on its largest video segmentation dataset dubbed SA-V dataset.
Meta stated the AI mannequin may also help ease the method of video enhancing or AI-based video era, in addition to to energy new experiences within the firm’s mixed-reality ecosystem. The thing monitoring functionality in movies may help in sooner annotation of visible knowledge to coach different laptop imaginative and prescient methods, the corporate added.
Since it’s an open-source AI mannequin, the corporate has hosted its weights on its GitHub web page. people can obtain and take a look at out the AI mannequin. Notably, it’s licenced beneath the Apache 2.zero licence which permits for analysis, tutorial, and non-commercial utilization.
