Building powerful AI-driven experiences with Jetpack Compose, Gemini and CameraX

Building powerful AI-driven experiences with Jetpack Compose, Gemini and CameraX

Home » News » Building powerful AI-driven experiences with Jetpack Compose, Gemini and CameraX
Table of Contents

The Android bot is a beloved mascot for Android customers and builders, with earlier variations of the bot builder being very talked-about – we determined that this yr we’d rebuild the bot maker from the bottom up, utilizing the newest expertise backed by Gemini. Immediately we’re releasing a brand new open supply app, Androidify, for studying easy methods to construct highly effective AI pushed experiences on Android utilizing the newest applied sciences corresponding to Jetpack Compose, Gemini by means of Firebase, CameraX, and Navigation 3.

Right here’s an instance of the app operating on the machine, showcasing changing a photograph to an Android bot that represents my likeness:

moving image showing the conversion of an image of a woman in a pink dress holding na umbrella into a 3D image of a droid bot wearing a pink dress holding an umbrella

Underneath the hood

The app combines a wide range of totally different Google applied sciences, corresponding to:

    • Gemini API – by means of Firebase AI Logic SDK, for accessing the underlying Imagen and Gemini fashions.
    • Jetpack Compose – for constructing the UI with pleasant animations and making the app adapt to totally different display screen sizes.
    • Navigation 3 – the newest navigation library for build up Navigation graphs with Compose.
    • CameraX Compose and Media3 Compose – for build up a customized digicam with customized UI controls (rear digicam assist, zoom assist, tap-to-focus) and taking part in the promotional video.

This pattern app is presently utilizing a normal Imagen mannequin, however we have been engaged on a fine-tuned mannequin that is skilled particularly on all the items that make the Android bot cute and enjoyable; we’ll share that model later this yr. Within the meantime, do not be stunned if the pattern app places out some attention-grabbing trying examples!

How does the Androidify app work?

The app leverages our greatest practices for Structure, Testing, and UI to showcase an actual world, trendy AI software on machine.

Flow chart describing Androidify app flow

Androidify app circulation chart detailing how the app works with AI

AI in Androidify with Gemini and ML Equipment

The Androidify app makes use of the Gemini fashions in a mess of how to counterpoint the app expertise, all powered by the Firebase AI Logic SDK. The app makes use of Gemini 2.5 Flash and Imagen 3 underneath the hood:

    • Picture validation: We make sure that the captured picture incorporates adequate info, corresponding to a clearly targeted particular person, and assessing for security. This function makes use of the multi-modal capabilities of Gemini API, by giving it a immediate and picture on the similar time:

val response = generativeModel.generateContent(
   content material {
       textual content(immediate)
       picture(picture)
   },
)

    • Textual content immediate validation: If the person opts for textual content enter as a substitute of picture, we use Gemini 2.5 Flash to make sure the textual content incorporates a sufficiently descriptive immediate to generate a bot.

    • Picture captioning: As soon as we’re positive the picture has sufficient info, we use Gemini 2.5 Flash to carry out picture captioning., We ask Gemini to be as descriptive as attainable,specializing in the clothes and its colours.

    • “Assist me write” function: Much like an “I’m feeling fortunate” sort function, “Assist me write” makes use of Gemini 2.5 Flash to create a random description of the clothes and coiffure of a bot.

    • Picture era from the generated immediate: As the ultimate step, Imagen generates the picture, offering the immediate and the chosen pores and skin tone of the bot.

The app additionally makes use of the ML Equipment pose detection to detect an individual within the viewfinder and allow the seize button when an individual is detected, in addition to including enjoyable indicators across the content material to point detection.

Discover extra detailed details about AI utilization in Androidify.

Jetpack Compose

The person interface of Androidify is constructed utilizing Jetpack Compose, the trendy UI toolkit that simplifies and accelerates UI growth on Android.

Pleasant particulars with the UI

The app makes use of Materials 3 Expressive, the newest alpha launch that makes your apps extra premium, fascinating, and interesting. It offers pleasant bits of UI out-of-the-box, like new shapes, componentry, and utilizing the MotionScheme variables wherever a movement spec is required.

MaterialShapes are utilized in numerous places. These are a preset listing of shapes that permit for simple morphing between one another—for instance, the lovable cookie form for the digicam seize button:

Androidify app UI showing camera button

Digicam button with a MaterialShapes.Cookie9Sided form

Past utilizing the usual Materials parts, Androidify additionally options customized composables and pleasant transitions tailor-made to the particular wants of the app:

    • There are many shared ingredient transitions throughout the app—for instance, a morphing form shared ingredient transition is carried out between the “take a photograph” button and the digicam floor.

      moving example of expressive button shapes in slow motion

    • Customized enter transitions for the ResultsScreen with the utilization of marquee modifiers.

      animated marquee example

    • Enjoyable colour splash animation as a transition between screens.

      moving image of a blue color splash transition between Androidify demo screens

    • Animating gradient buttons for the AI-powered actions.

      animated gradient button for AI powered actions example

To be taught extra in regards to the distinctive particulars of the UI, learn Androidify: Constructing pleasant UIs with Compose

Adapting to totally different units

Androidify is designed to look nice and performance seamlessly throughout sweet bar telephones, foldables, and tablets. The final aim of growing adaptive apps is to keep away from reimplementing the identical app a number of instances on every type issue by extracting out reusable composables, and leveraging APIs like WindowSizeClass to find out what sort of structure to show.

a collage of different adaptive layouts for the Androidify app across small and large screens

Numerous adaptive layouts within the app

For Androidify, we solely wanted to leverage the width window measurement class. Combining this with totally different structure mechanisms, we have been capable of reuse or prolong the composables to cater to the multitude of various machine sizes and capabilities.

    • Responsive layouts: The CreationScreen demonstrates adaptive design. It makes use of helper features like isAtLeastMedium() to detect window measurement classes and modify its structure accordingly. On bigger home windows, the picture/immediate space and colour picker would possibly sit side-by-side in a Row, whereas on smaller home windows, the colour picker is accessed by way of a ModalBottomSheet. This sample, referred to as “supporting pane”, highlights the supporting dependencies between the principle content material and the colour picker.

    • Foldable assist: The app actively checks for foldable machine options. The digicam display screen makes use of WindowInfoTracker to get FoldingFeature info to adapt to totally different options by optimizing the structure for tabletop posture.

    • Rear show: Help for units with a number of shows is included by way of the RearCameraUseCase, permitting for the machine digicam preview to be proven on the exterior display screen when the machine is unfolded (so the principle content material is often displayed on the interior display screen).

Utilizing window measurement lessons, coupled with making a customized @LargeScreensPreview annotation, helps obtain distinctive and helpful UIs throughout the spectrum of machine sizes and window sizes.

CameraX and Media3 Compose

To permit customers to base their bots on images, Androidify integrates CameraX, the Jetpack library that makes digicam app growth simpler.

The app makes use of a customized CameraLayout composable that helps the structure of the everyday composables {that a} digicam preview display screen would come with— for instance, zoom buttons, a seize button, and a flip digicam button. This structure adapts to totally different machine sizes and extra superior use instances, just like the tabletop mode and rear-camera show. For the precise rendering of the digicam preview, it makes use of the brand new CameraXViewfinder that’s a part of the camerax-compose artifact.

CameraLayout in Compose

CameraLayout composable that takes care of various machine configurations, corresponding to desk prime mode

CameraLayout in Compose

CameraLayout composable that takes care of various machine configurations, corresponding to desk prime mode

The app additionally integrates with Media3 APIs to load an tutorial video for exhibiting easy methods to get one of the best bot from a immediate or picture. Utilizing the brand new media3-ui-compose artifact, we will simply add a VideoPlayer into the app:

@Composable
personal enjoyable VideoPlayer(modifier: Modifier = Modifier) {
    val context = LocalContext.present
    var participant by keep in mind { mutableStateOf<Participant?>(null) }
    LifecycleStartEffect(Unit) {
        participant = ExoPlayer.Builder(context).construct().apply {
            setMediaItem(MediaItem.fromUri(Constants.PROMO_VIDEO))
            repeatMode = Participant.REPEAT_MODE_ONE
            put together()
        }
        onStopOrDispose {
            participant?.launch()
            participant = null
        }
    }
    Field(
        modifier
            .background(MaterialTheme.colorScheme.surfaceContainerLowest),
    ) {
        participant?.let { currentPlayer ->
            PlayerSurface(currentPlayer, surfaceType = SURFACE_TYPE_TEXTURE_VIEW)
        }
    }
}

Utilizing the brand new onLayoutRectChanged modifier, we additionally hear for whether or not the composable is totally seen or not, and play or pause the video primarily based on this info:

var videoFullyOnScreen by keep in mind { mutableStateOf(false) }     

LaunchedEffect(videoFullyOnScreen) {
     if (videoFullyOnScreen) currentPlayer.play() else currentPlayer.pause()
} 

// We add this onto the participant composable to find out if the video composable is seen, and mutate the videoFullyOnScreen variable, that then toggles the participant state. 
Modifier.onVisibilityChanged(
                containerWidth = LocalView.present.width,
                containerHeight = LocalView.present.peak,
) { fullyVisible -> videoFullyOnScreen = fullyVisible }

// A easy model of visibility modified detection
enjoyable Modifier.onVisibilityChanged(
    containerWidth: Int,
    containerHeight: Int,
    onChanged: (seen: Boolean) -> Unit,
) = this then Modifier.onLayoutRectChanged(100, 0) { layoutBounds ->
    onChanged(
        layoutBounds.boundsInRoot.prime > 0 &&
            layoutBounds.boundsInRoot.backside < containerHeight &&
            layoutBounds.boundsInRoot.left > 0 &&
            layoutBounds.boundsInRoot.proper < containerWidth,
    )
}

Moreover, utilizing rememberPlayPauseButtonState, we add on a layer on prime of the participant to supply a play/pause button on the video itself:

val playPauseButtonState = rememberPlayPauseButtonState(currentPlayer)
            OutlinedIconButton(
                onClick = playPauseButtonState::onClick,
                enabled = playPauseButtonState.isEnabled,
            ) {
                val icon =
                    if (playPauseButtonState.showPlay) R.drawable.play else R.drawable.pause
                val contentDescription =
                    if (playPauseButtonState.showPlay) R.string.play else R.string.pause
                Icon(
                    painterResource(icon),
                    stringResource(contentDescription),
                )
            }

Take a look at the code for extra particulars on how CameraX and Media3 have been utilized in Androidify.

Navigation 3

Display transitions are dealt with utilizing the brand new Jetpack Navigation 3 library androidx.navigation3. The MainNavigation composable defines the totally different locations (Dwelling, Digicam, Creation, About) and shows the content material related to every vacation spot utilizing NavDisplay. You get full management over your again stack, and navigating to and from locations is so simple as including and eradicating gadgets from a listing.

@Composable
enjoyable MainNavigation() {
   val backStack = rememberMutableStateListOf<NavigationRoute>(Dwelling)
   NavDisplay(
       backStack = backStack,
       onBack = { backStack.removeLastOrNull() },
       entryProvider = entryProvider {
           entry<Dwelling> { entry ->
               HomeScreen(
                   onAboutClicked = {
                       backStack.add(About)
                   },
               )
           }
           entry<Digicam> {
               CameraPreviewScreen(
                   onImageCaptured = { uri ->
                       backStack.add(Create(uri.toString()))
                   },
               )
           }
           // and so on
       },
   )
}

Notably, Navigation 3 exposes a brand new composition native, LocalNavAnimatedContentScope, to simply combine your shared ingredient transitions without having to maintain monitor of the scope your self. By default, Navigation 3 additionally integrates with predictive again, offering pleasant again experiences when navigating between screens, as seen on this prior shared ingredient transition:

CameraLayout in Compose

Study extra about Jetpack Navigation 3, presently in alpha.

Study extra

By combining the declarative energy of Jetpack Compose, the digicam capabilities of CameraX, the clever options of Gemini, and considerate adaptive design, Androidify is a personalised avatar creation expertise that feels proper at residence on any Android machine. You could find the complete code pattern at github.com/android/androidify the place you may see the app in motion and be impressed to construct your individual AI-powered app experiences.

Discover this announcement and all Google I/O 2025 updates on io.google beginning Could 22.

Supply hyperlink

author avatar
roosho Senior Engineer (Technical Services)
I am Rakib Raihan RooSho, Jack of all IT Trades. You got it right. Good for nothing. I try a lot of things and fail more than that. That's how I learn. Whenever I succeed, I note that in my cookbook. Eventually, that became my blog. 
share this article.

Enjoying my articles?

Sign up to get new content delivered straight to your inbox.

Please enable JavaScript in your browser to complete this form.
Name