React is generally coded in .jsx files, which need to be compiled/transpiled into regular Javascript. Once this is done, you can serve these files to web browsers any way you like.
Understanding Build tools
React compilation/transpilation is done using Webpack or Parcel. If you're using Create-React-App, then you're using Webpack under the hood. CRA npm start will run a Webpack dev server, and npm run build will put all of the files you need in your build directory. You should do the latter if you want to
Avoid using Webpack dev server
Most tutorials, including the one linked in the comments, suggest using Webpack development server (aka npm start) to run your server in Docker. which is not recommended. Not only does it consume more resources to run, but the development build will run slower in browsers than the production build. The development server is also not meant to handle heavy loads.
Use build for production, with a real server
Use npm run build to generate the build files in the build directory, then use a real web server to serve them to the world.
One example in Docker, using Nginx to serve the files, can be found here
Alternatively the CRA build command itself offers a helpful hint:
The build folder is ready to be deployed.
You may serve it with a static server:
npm install -g serve
serve -s build
This will work in a Docker Node container. An example of a very not-optimized Dockerfile might look like this
FROM node:16-alpine
COPY . . 
RUN npm install
RUN npm build
RUN npm install -D serve
CMD ["./node_modules/.bin/serve", "-s", "build"]
The build process can be improved here to leverage cache (by first copying package.json and package-lock.json and installing before copying code over, see example at the end of the answer here), and all of the RUN commands combined