You should not create instance of B in class A - instead you should provide instance B when you are creating A - it's called Inversion of Control. Popular way to do it is with dependency injection, for example with Spring framework.
Then everything becomes simple - when your class needs to read a file with FileReader class, for tests you can create a 'fake' implementation, for example when you call FileReader.readFile(), your fake implementation will simply return hardcoded String, or Stream, depends what you want.
This concept is really big, imagine when your FileReader is actually a DatabaseReader or ExternalServiceCaller. Instead of testing real database in unit tests (good luck with that), you create a FakeDatabaseReader that operates on regular Java HashMap and everything is easy to test.
Or when your code deals with times/dates, imagine a functionality that does something only at 29th February - you could wait for 4 years to test it, or instead provide a Clock object to your test that is set to particular date. The Clock class has static methods offering a variety of alternative clocks, to provide a fixed moment, a moment adjusted into the future or the past, or a clock with an altered cadence.