Adobe is continuing on its Firefly AI flight path. First, it introduced Firefly AI-powered features in beta versions of Illustrator, Express, and Photoshop, and now those products are in general release. Also, Firefly is now available as a stand-alone Web application. And, with IBC 2023 just around the corner, Adobe announced new beta versions of its video products—Premiere Pro, After Effects, and Frame.io—adding numerous new features and functionality, including some big enhancements powered by AI.
What do we think? Adobe clearly has been focused on AI development for a while now, and while it is far ahead in its work of integrating AI-powered workflow in its products, the company is carefully navigating the potential minefield associated with AI technology. Just yesterday, Adobe was one of eight additional tech companies to support AI safeguard agreements with the White House. Adobe also has been active in taking steps to train its datasets responsibly, to tag AI-created work, and more.
Adobe is one of the largest content creation tool vendors, and the DCC market is booming, as demand for digital content across all platforms drives double-digit growth, as discussed in detail in JPR’s 2023 Digital Content Creation report. But now that AI is creeping into the DCC workflow, this adds a new dimension to the DCC market, a situation addressed in an AI addendum to the report, which addresses how AI will help creators ply their trade. According to the report, AI will be used for repetitive, non-creative tasks such as rotoscoping. So, we’re not surprised about some of the new features Adobe has integrated into the new beta version of After Effects, supporting that assessment in the report.
Adobe unshackles its initial Firefly-powered products, introduces new video offerings into beta
As they say, you can’t put the genie back in the bottle. The same can be said about AI, because once content creators became aware of the power it can bestow on them to create, it is too magical to give up. Such is the case with Adobe Firefly, which infuses AI features into Adobe’s products.
Adobe began its AI journey back in March 2023, when it introduced a family of AI models it coined “Firefly,” making them available via a variety of models. Then, from April to June, the company sprinkled Firefly’s magic into beta versions of Illustrator, Express, and Photoshop, the latter of which uses natural language prompts to create images with generative fill, to add content, remove or replace parts of an image, extend the edges of an image, and more. It’s worth pointing out that Firefly AI-powered workflows are nondestructive, creating a new layer without adversely affecting the underlying asset within Photoshop or Illustrator. Then, in July, the company expanded and diversified Firefly to support over 100 languages. Now, Firefly is ready to fly on its own, as Adobe has made all the betas of the latest Express, Photoshop, and Illustrator versions, which are infused with AI, generally available starting today. In Express, the generative AI (GA) of Firefly modules include text effects and text to image. In the Creative Cloud flagship apps, this entails the GA generative fill and generative expand in Photoshop (desktop and Web), and GA generative recolor in Illustrator.
Adobe has also made Firefly a stand-alone Web application, with current modules including text to image, text effects, generative fill, and generative recolor.
Adobe said that when it introduced Firefly back in March 2023, it was launched as a family of creative generative AI models, and that has not changed, as the training was done on unique datasets designed to be commercially safe and not exploitive of artists. Furthermore, anything created in Firefly can now be used for commercial purposes, something that was not permitted in the beta versions.
According to Adobe, it has plans to bring more Firefly-powered workflows into the ecosystem. And those plans are moving quickly. Today, ahead of IBC 2023, Adobe announced updates to its video offerings—Premiere Pro, After Effects, and Frame.io—including Firefly AI integration in beta releases of Premiere Pro and After Effects, as well as the addition of 3D- and API-backed features.
Premiere Pro
New AI-powered features have been added to Premiere Pro (beta), augmenting the text-based editing that was launched in beta this past Maya, enabling users to create rough cuts by copying and pasting text generated by AI speech to text. Now, the text-based editing has been improved and includes the addition of filler word detection, which automatically identifies “uhs” and “ums” and deletes the filler words with a single click, with those edits automatically appearing in the sequence. Pauses can also be deleted, and users have the ability to set the duration of the pauses they would like deleted, starting at the minimum setting of a tenth of a second.
The second AI feature in Premiere Pro is Enhance Speech, which uses Adobe Sensei to remove noise from voice recordings and improve poorly recorded dialogue, again with one click. AI automatically categorizes audio and makes voices sound as if they were recorded in a professional studio. With the mix slider, users can add the desired degree of background noise into the recording. Also, the new Audio Category Tagging feature uses AI to identify which clips have dialogue, which have music, and which have sound effects or ambient noise. Enhance Speech runs completely offline and does not require an Internet connection.
In addition, improvements to Premiere Pro’s timeline has made it 5× faster, making editing, trimming, and other timeline-related tasks faster and more responsive. System performance can be optimized with the new Effects Manager, which finds incompatible plug-ins. Also, Adobe has improved color, too, with new color preferences; three new tone-mapping methods have been added, too, along with better LUT management and relinking, and a QuickTime gamma fix that matches the sequence gamma setting to that in QuickTime, ensuring the contrast and color in the video looks the same in Premiere Pro as it does in QuickTime.
Adobe called this the fastest and most reliable release it has ever shipped (don’t companies always say that?). There does seems to be some validity to that statement, however. As for the former, that is accomplished through AI and the speed at which the timeline UI redraws. As for the latter, Adobe has added features to keep user systems running optimally, including the ability to detect bad third-party plug-ins and a project recovery feature so users do not lose work in the event of a system crash. There is also a background auto save function.
After Effects
Rotoscoping has always been a labor-intensive process, but next-gen AI rotoscoping capabilities in the Roto Brush tools of After Effects streamlines workflows by removing objects in a faster and more accurate way. This includes hard-to-isolate objects like overlapping limbs, hair, and transparencies. This feature alone will save artists hours of tedious work.
Adobe also introduced a new, true 3D workspace to After Effects for integrating, animating, lighting, shading, and rendering 3D models. Models can also be combined with 2D elements for adding depth and dimension to composites, all within a unified space. The space supports image-based lighting, enabling users to place models into a scene with complementary lighting and shadows, in a more physically-accurate, photorealistic way through a new advanced 3D engine that is fully GPU accelerated.
New features have been added to Frame.io, including refreshed review and approval tools for artists who need to compare a wide range of assets. For them, the comparison viewer now supports viewing video, audio, photos, design files, and PDF assets side-by-side, with commenting and annotation available.
There’s also more support for Camera to Cloud, with five new connections for Frame.io and Creative Cloud customers.
The all-new Frame.io Storage Connect lets users connect their own AWS S3 bucket directly to Frame.io, thereby reducing storage costs up to 60%, according to the company. Storage Connect is included and bundled into the existing price of Frame.io Enterprise, making this workflow available to Enterprise-level customers only.
All the new features for the video offerings are available now in beta and will be generally available later this fall.