4

This question is a follow-up question to Custom object drag-and-drop from FX to Swing.

I'm working on a plugin for a Swing application that uses JavaFX for some graphical user interfaces. We added drag-and-drop functionality to improve the user experience. First, we were using an external JavaFX window (Stage) for our Scene, now we want to embed it directly into the Swing application via a JFXPanel.

Now, the strange thing is, that it seems to make a big difference for drag-and-drop whether the exactly same Scene is loaded in a Stage or in a JFXPanel.

I already encountered some problems when trying to drag some custom Java object (in serialized form) with a custom MIME type from a JavaFX application into a Swing application. However, my problems were solved in the question I mentioned above. Now, using the embedded JavaFX application, I encounter some new problems, so I wanted to ask if someone had similar problems or knows a solution for this scenario.

I've written a MVCE, it's a simple Java application with a drag-supporting JFXPanel on the one side and a drop-supporting JPanel on the other side:

public class MyApp {

    public static final DataFormat FORMAT = new DataFormat(
        // this works fine in a separate window
        //"JAVA_DATAFLAVOR:application/x-my-mime-type; class=java.lang.String",
        "application/x-my-mime-type; class=java.lang.String");

    public static final DataFlavor FLAVOR;

    static {
        try {
            FLAVOR = new DataFlavor("application/x-my-mime-type; class=java.lang.String");
        } catch (ClassNotFoundException ex) {
            throw new RuntimeException(ex);
        }
    }

    public static void main(String[] args) {
        new MyApp().run();
    }

    private void run() {
        JFrame frame = new JFrame();
        frame.setLayout(new GridLayout(1, 2));
        frame.add(buildFX());
        frame.add(buildSwing());
        frame.setSize(300, 300);
        frame.setVisible(true);
    }

    private JFXPanel buildFX() {
        BorderPane parent = new BorderPane();
        parent.setOnDragDetected(event -> {
            Dragboard dragboard = parent.startDragAndDrop(TransferMode.COPY);
            ClipboardContent content = new ClipboardContent();
            content.put(FORMAT, "Test");
            dragboard.setContent(content);
            event.consume();
        });
        JFXPanel panel = new JFXPanel();
        panel.setScene(new Scene(parent));
        return panel;
    }

    @SuppressWarnings("serial")
    private JPanel buildSwing() {
        JPanel panel = new JPanel();
        panel.setBackground(Color.ORANGE);
        panel.setTransferHandler(new TransferHandler() {

            @Override
            public boolean canImport(TransferSupport support) {
                return support.isDataFlavorSupported(FLAVOR);
            }

            @Override
            public boolean importData(TransferSupport support) {
                if (!canImport(support)) return false;
                try {
                    String data = (String) support.getTransferable().getTransferData(FLAVOR);
                    System.out.println(data);
                    return true;
                } catch (UnsupportedFlavorException e) {
                    e.printStackTrace();
                } catch (IOException e) {
                    e.printStackTrace();
                }
                return false;
            }

        });
        return panel;
    }
}

According to the answer in the other question, using the prefix JAVA_DATAFLAVOR: in the DataFormat is necessary for Swing to handle the MIME type correctly. However, when using such a DataFormat inside a JFXPanel (disabled in the example), it seems like Java tries to construct a DataFlavor when dragging from the FX application and fails to parse the MIME type with the prefix:

Exception in thread "AWT-EventQueue-0" java.lang.IllegalArgumentException: failed to parse:JAVA_DATAFLAVOR:application/x-my-mime-type; class=java.lang.String
    at java.awt.datatransfer.DataFlavor.<init>(Unknown Source)
    at javafx.embed.swing.SwingDnD$DnDTransferable.getTransferDataFlavors(SwingDnD.java:394)
    at sun.awt.datatransfer.DataTransferer.getFormatsForTransferable(Unknown Source)
    at sun.awt.dnd.SunDragSourceContextPeer.startDrag(Unknown Source)
    at java.awt.dnd.DragSource.startDrag(Unknown Source)
    at java.awt.dnd.DragSource.startDrag(Unknown Source)
    at java.awt.dnd.DragGestureEvent.startDrag(Unknown Source)
    at javafx.embed.swing.SwingDnD.startDrag(SwingDnD.java:280)
    at javafx.embed.swing.SwingDnD.lambda$null$66(SwingDnD.java:247)
    at java.awt.event.InvocationEvent.dispatch(Unknown Source)
    at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
    at java.awt.EventQueue.access$500(Unknown Source)
    at java.awt.EventQueue$3.run(Unknown Source)
    at java.awt.EventQueue$3.run(Unknown Source)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
    at java.awt.EventQueue.dispatchEvent(Unknown Source)
    at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
    at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
    at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
    at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
    at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
    at java.awt.EventDispatchThread.run(Unknown Source)

Using only the pure MIME type, without the prefix, the drag-and-drop operation works and I can even receive the correct DataFlavor (java.awt.datatransfer.DataFlavor[mimetype=application/x-my-mime-type;representationclass=java.lang.String]), but the dropped data is always null. As seen in the other question, using this second approach with two separated windows, I can't even receive the DataFlavor, but now it works somehow to this limited point.

Lukas Körfer
  • 13,515
  • 7
  • 46
  • 62

1 Answers1

3

Probably there is a bit of misunderstanding of how the transfer works.

Trying to retrieve the transfer data directly as a string may work for "text/plain" or other standard text types and, as you note, with some quirks for particular cases of custom unregistered type. But I don't think the effort for custom workarounds is justified.

Since you control entirely the content structure for the custom mime type and both ends of the data producer and the consumer in the same application, I suggest not to deal with internal toolkit implementation-dependent prefixes or class mappings. Probably better is just to define your MIME type without unrelated metadata and malformed prefixes (as it is supposed to be).

Defining an "application/x-my-mime" type and correctly decoding the data should be enough.

Sample 1 (serialized data)

The below, corrected from your sample, should drop the data fine to the Swing frame in Java 8. 

package jfxtest;

import java.awt.Color;
import java.awt.GridLayout;
import java.awt.datatransfer.DataFlavor;
import java.io.InputStream;
import java.io.ObjectInputStream;
import java.util.Collections;
import javafx.embed.swing.JFXPanel;
import javafx.scene.Node;
import javafx.scene.Scene;
import javafx.scene.input.DataFormat;
import javafx.scene.input.TransferMode;
import javafx.scene.layout.BorderPane;
import javax.swing.JFrame;
import javax.swing.JPanel;
import javax.swing.SwingUtilities;
import javax.swing.TransferHandler;
import javax.swing.TransferHandler.TransferSupport;

public class MyApp {

  final static String MY_MIME_TYPE = "application/x-my-mime";
  public static final DataFormat FORMAT = new DataFormat(MY_MIME_TYPE);
  public static final DataFlavor FLAVOR = new DataFlavor(MY_MIME_TYPE, "My Mime Type");

  private void startDrag(Node node) {
    node.startDragAndDrop(TransferMode.COPY).setContent(
        Collections.singletonMap(FORMAT, "Test"));
  }

  private boolean processData(TransferSupport support) {
    try (InputStream in = (InputStream) support.getTransferable().getTransferData(FLAVOR)) {
      Object transferred = new ObjectInputStream(in).readObject();
      System.out.println("transferred: " + transferred + " (" + transferred.getClass() + ")");
      return true;
    } catch (Exception e) {
      e.printStackTrace();
    }
    return false;
  }  

  public static void main(String[] args) {
    new MyApp().run();
  }

  private void run() {
    JFrame frame = new JFrame();
    frame.setLayout(new GridLayout(1, 2));
    frame.add(buildSwing());
    SwingUtilities.invokeLater(() -> {
      frame.add(buildFX());
    });
    frame.setSize(300, 300);
    frame.setVisible(true);
    frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
  }

  private JFXPanel buildFX() {
    BorderPane parent = new BorderPane();
    parent.setOnDragDetected(event -> {
      startDrag(parent);
      event.consume();
    });
    JFXPanel panel = new JFXPanel();
    panel.setScene(new Scene(parent));
    return panel;
  }


  private JPanel buildSwing() {
    JPanel panel = new JPanel();
    panel.setBackground(Color.ORANGE);
    panel.setTransferHandler(new TransferHandler() {
      private static final long serialVersionUID = 1L;

      @Override
      public boolean canImport(TransferSupport support) {
        return support.isDataFlavorSupported(FLAVOR);
      }

      @Override
      public boolean importData(TransferSupport support) {
        if (canImport(support)) {
          return processData(support);
        }
        return false;
      }

    });
    return panel;
  }

}

Output: transferred: Test (class java.lang.String)

The essential excerpt here is:

...

  final static String MY_MIME_TYPE = "application/x-my-mime";
  public static final DataFormat FORMAT = new DataFormat(MY_MIME_TYPE);
  public static final DataFlavor FLAVOR = new DataFlavor(MY_MIME_TYPE, "My Mime Type");

  private void startDrag(Node node) {
    node.startDragAndDrop(TransferMode.COPY).setContent(
        Collections.singletonMap(FORMAT, "Test"));    
  }

  private boolean processData(TransferSupport support) {
    try (InputStream in = (InputStream) support.getTransferable().getTransferData(FLAVOR)) {
      Object transferred = new ObjectInputStream(in).readObject();
      System.out.println("transferred: " + transferred + " (" + transferred.getClass() + ")");
      return true;
    } catch (Exception e) {
      e.printStackTrace();
    }
    return false;
  }

...

Note, the data retrieval is simplistic for illustration purposes, for a real application one probably would like to add a more rigorous stream reading, handling errors, etc.

Sample 2 (custom mime with text)

The first sample transfers a serialized object (which is usually a good and simple thing, as you can transfer anything serializable, but makes it hard to transfer/accept, say, 3rd party JSON). In the unlikely case when you wish to produce real text or other arbitrary content for the custom MIME instead of a serialized object, the below should do the job:

package jfxtest;

import java.awt.Color;
import java.awt.GridLayout;
import java.awt.datatransfer.DataFlavor;
import java.io.InputStream;
import java.io.ObjectInputStream;
import java.nio.charset.StandardCharsets;
import java.util.Collections;
import javafx.embed.swing.JFXPanel;
import javafx.scene.Node;
import javafx.scene.Scene;
import javafx.scene.input.DataFormat;
import javafx.scene.input.TransferMode;
import javafx.scene.layout.BorderPane;
import javax.swing.JFrame;
import javax.swing.JPanel;
import javax.swing.SwingUtilities;
import javax.swing.TransferHandler;
import javax.swing.TransferHandler.TransferSupport;

public class MyApp {

  final static String MY_MIME_TYPE = "application/x-my-mime";
  public static final DataFormat FORMAT = new DataFormat(MY_MIME_TYPE);
  public static final DataFlavor FLAVOR = new DataFlavor(MY_MIME_TYPE, "My Mime Type");

  private void startDrag(Node node) {
    node.startDragAndDrop(TransferMode.COPY).setContent(
        // put a ByteBuffer to transfer the content unaffected
        Collections.singletonMap(FORMAT, StandardCharsets.UTF_8.encode("Test")));
  }

  private boolean processData(TransferSupport support) {
    try (InputStream in = (InputStream) support.getTransferable().getTransferData(FLAVOR)) {
      byte[] textBytes = new byte[in.available()];
      in.read(textBytes);
      String transferred = new String(textBytes, StandardCharsets.UTF_8); 
      System.out.println("transferred text: " + transferred);
      return true;
    } catch (Exception e) {
      e.printStackTrace();
    }
    return false;
  }  

  public static void main(String[] args) {
    new MyApp().run();
  }

  private void run() {
    JFrame frame = new JFrame();
    frame.setLayout(new GridLayout(1, 2));
    frame.add(buildSwing());
    SwingUtilities.invokeLater(() -> {
      frame.add(buildFX());
    });
    frame.setSize(300, 300);
    frame.setVisible(true);
    frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
  }

  private JFXPanel buildFX() {
    BorderPane parent = new BorderPane();
    parent.setOnDragDetected(event -> {
      startDrag(parent);
      event.consume();
    });
    JFXPanel panel = new JFXPanel();
    panel.setScene(new Scene(parent));
    return panel;
  }


  private JPanel buildSwing() {
    JPanel panel = new JPanel();
    panel.setBackground(Color.ORANGE);
    panel.setTransferHandler(new TransferHandler() {
      private static final long serialVersionUID = 1L;

      @Override
      public boolean canImport(TransferSupport support) {
        return support.isDataFlavorSupported(FLAVOR);
      }

      @Override
      public boolean importData(TransferSupport support) {
        if (canImport(support)) {
          return processData(support);
        }
        return false;
      }

    });
    return panel;
  }

}

Output: transferred text: Test

The essential part here is:

...

  private void startDrag(Node node) {
    node.startDragAndDrop(TransferMode.COPY).setContent(
        // put a ByteBuffer to transfer the content unaffected
        Collections.singletonMap(FORMAT, StandardCharsets.UTF_8.encode("Test")));
  }

  private boolean processData(TransferSupport support) {
    try (InputStream in = (InputStream) support.getTransferable().getTransferData(FLAVOR)) {
      byte[] textBytes = new byte[in.available()];
      in.read(textBytes);
      String transferred = new String(textBytes, StandardCharsets.UTF_8); 
      System.out.println("transferred text: " + transferred);
      return true;
    } catch (Exception e) {
      e.printStackTrace();
    }
    return false;
  }  

...

Once again, being an illustration, the stream, error, etc. handling here is simplistic.

One thing to note is that there is also a predefined "application/x-java-serialized-object" (DataFlavor.javaSerializedObjectMimeType) for the more generic and easier deserialization. But long-term custom MIME seems more flexible and more straightforward to handle overall.

Fedor Losev
  • 3,244
  • 15
  • 13
  • 1
    Nice and detailed answer, that helped me solve my problem. I did not think about the manual deserialization with a default MIME type, because the solution with the prefixed Java MIME type (`JAVA_DATAFLAVOR:`) from the other question worked really well. – Lukas Körfer Oct 08 '18 at 13:40