To make animations work in Compose you need to animate a value of some particular modifier. There's no way how you can animate between different set of modifiers.
Following this documentation paragraph, you can animate value for Modifier.size.
First I wait for the size of the image to be determined, with this value the size modifier can be set (I use then with an empty Modifier before that) and then this value can be animated.
Here's a sample:
Column {
    val animatableSize = remember { Animatable(Size.Zero, Size.VectorConverter) }
    val (containerSize, setContainerSize) = remember { mutableStateOf<Size?>(null) }
    val (imageSize, setImageSize) = remember { mutableStateOf<Size?>(null) }
    val density = LocalDensity.current
    val scope = rememberCoroutineScope()
    Button(onClick = {
        scope.launch {
            if (imageSize == null || containerSize == null) return@launch
            val targetSize = if (animatableSize.value == imageSize) containerSize else imageSize
            animatableSize.animateTo(
                targetSize,
                animationSpec = tween(durationMillis = 1000)
            )
        }
    }) {
        Text("Animate")
    }
    Box(
        Modifier
            .padding(20.dp)
            .size(300.dp)
            .background(Color.LightGray)
            .onSizeChanged { size ->
                setContainerSize(size.toSize())
            }
    ) {
        Image(
            Icons.Default.Person,
            contentDescription = null,
            modifier = Modifier
                .then(
                    if (animatableSize.value != Size.Zero) {
                        animatableSize.value.run {
                            Modifier.size(
                                width = with(density) { width.toDp() },
                                height = with(density) { height.toDp() },
                            )
                        }
                    } else {
                        Modifier
                    }
                )
                .onSizeChanged { intSize ->
                    if (imageSize != null) return@onSizeChanged
                    val size = intSize.toSize()
                    setImageSize(size)
                    scope.launch {
                        animatableSize.snapTo(size)
                    }
                }
        )
    }
}
Result:
