To customize each fragment's tool bar individually, I have created a custom tool bar and using it inside each fragment's layout.
common_toolbar.xml
<layout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto">
<com.google.android.material.appbar.AppBarLayout
    android:layout_width="match_parent"
    android:layout_height="wrap_content">
    <androidx.constraintlayout.widget.ConstraintLayout
        android:id="@+id/const_item"
        android:layout_width="match_parent"
        android:layout_height="?attr/actionBarSize">
        <TextView
            android:id="@+id/back_btn"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:layout_alignParentStart="true"
            android:layout_centerVertical="true"
            android:layout_gravity="center_vertical"
            android:layout_marginStart="@dimen/margin_normal"
            android:gravity="center"
            android:padding="@dimen/padding_five"
            android:text="Back"
            android:textColor="@android:color/white"
            android:textSize="@dimen/font_size_fifteen"
            app:drawableStartCompat="@drawable/ic_action_chevron_left"
            app:drawableTint="@android:color/white"
            app:layout_constraintBottom_toBottomOf="parent"
            app:layout_constraintStart_toStartOf="parent"
            app:layout_constraintTop_toTopOf="parent" />
        <TextView
            android:id="@+id/title_toolbar"
            android:layout_width="wrap_content"
            android:layout_height="match_parent"
            android:layout_centerHorizontal="true"
            android:gravity="center"
            android:text="@string/menu"
            android:textColor="@android:color/white"
            android:textSize="@dimen/font_size_fifteen"
            app:layout_constraintBottom_toBottomOf="parent"
            app:layout_constraintEnd_toEndOf="parent"
            app:layout_constraintStart_toStartOf="parent"
            app:layout_constraintTop_toTopOf="parent" />
        
    </androidx.constraintlayout.widget.ConstraintLayout>
</com.google.android.material.appbar.AppBarLayout>
And then adding it to fragments's UI. Let's say, adding to  
fragment_user_details.xml 
<layout xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:android="http://schemas.android.com/apk/res/android">
<androidx.constraintlayout.widget.ConstraintLayout
    android:layout_width="match_parent"
    android:layout_height="match_parent">
    
    <include layout="@layout/toolbar_common"
        android:id="@+id/toolbar_user_details"/>
    <!-- 
     Other View widgets
     -->
</androidx.constraintlayout.widget.ConstraintLayout>
As each fragment has its own tool bar, so I am getting it by it's id and customize as per my requirement. Make sure Activity has `Theme.MaterialComponents.Light.NoActionBar` theme.