I managed to get this working after a bit of looking around. There are two major pieces that had to be in place to make it work.
Use the appropriate loader
In this case I wanted to use the raw-loader, so I installed it as a dev dependency. yarn add -D raw-loader.
In order to actually import the file I needed to then override the webpack configuration like this:
// eslint-disable-next-line import/no-webpack-loader-syntax
import toolbarItems from '!!raw-loader!../ToolbarItems';
This loads the entire file into the variable toolbarItems. By using the !! before the loader I  prevent any other webpack loaders from processing it in this specific case. This might work on its own in a plain javascript project, but in typescript...
You must provide a module to typescript
I was running into the typescript error:
Failed to compile.
/Users/cory/Code/something/custom-theme/src/pages/NavigationDemo.tsx
TypeScript error in /Users/cory/Code/something/custom-theme/src/pages/NavigationDemo.tsx(9,26):
Cannot find module '!!raw-loader!../ToolbarItems'.  TS2307
     7 |
     8 | // eslint-disable-next-line import/no-webpack-loader-syntax
  >  9 | import toolbarItems from '!!raw-loader!../ToolbarItems';
       |                          ^
    10 |
    11 | const useStyles = makeStyles({
    12 |   root: {
Simply declaring a module for the loader in a file called ToolbarItems.ts.d.ts solved the issue for me:
declare module '!!raw-loader!*' {
  const content: string;
  export default content;
}
source