Not in a JTextArea, but in a JEditorPane or JTextPane you can use a StyledDocument.
On key down event of ^ (KeyEvent.VK_CIRCUMFLEX) you add an HTML <sup>...</sup> probably with attribute styles on the StyledDocument.
This requires some work, as opposed to the ready math solutions of @Schokokuchen_Bäcker.
You could provide an edit icon for ^ like [x], especially as one some national keyboards ^ is a dead key to create letters like û. Even more work.
A change of character attribute for superscript then corresponds to a ^ in your representation.
As JTextPane is a child class of JEditorPane, I would use that one.
JTextPane textPane = new JTextPane(document);
textPane.setEditable(true);
Element elememt = doc.getCharacterElement(docOffset);
if (StyleConstants.isSuperscript(element.getAttributes())) { ...
One might use SimpleAttributeSet to compose attributes.