In my project I managed to use GridLayout and results are very stable, with no flickering and with a perfectly working vertical scrollbar.
First I created a JPanel for the settings; in my case it is a grid with a row for each parameter and two columns: left column is for labels and right column is for components. I believe your case is similar.
JPanel yourSettingsPanel = new JPanel();
yourSettingsPanel.setLayout(new GridLayout(numberOfParams, 2));
I then populate this panel by iterating on my parameters and alternating between adding a JLabel and adding a component.
for (int i = 0; i < numberOfParams; ++i) {
yourSettingsPanel.add(labels[i]);
yourSettingsPanel.add(components[i]);
}
To prevent yourSettingsPanel from extending to the entire container I first wrap it in the north region of a dummy panel, that I called northOnlyPanel.
JPanel northOnlyPanel = new JPanel();
northOnlyPanel.setLayout(new BorderLayout());
northOnlyPanel.add(yourSettingsPanel, BorderLayout.NORTH);
Finally I wrap the northOnlyPanel in a JScrollPane, which should behave nicely pretty much anywhere.
JScrollPane scroll = new JScrollPane(northOnlyPanel,
JScrollPane.VERTICAL_SCROLLBAR_ALWAYS,
JScrollPane.HORIZONTAL_SCROLLBAR_NEVER);
Most likely you want to display this JScrollPane extended inside a JFrame; you can add it to a BorderLayout JFrame, in the CENTER region:
window.add(scroll, BorderLayout.CENTER);
In my case I put it on the left column of a GridLayout(1, 2) panel, and I use the right column to display contextual help for each parameter.
JTextArea help = new JTextArea();
help.setLineWrap(true);
help.setWrapStyleWord(true);
help.setEditable(false);
JPanel split = new JPanel();
split.setLayout(new GridLayout(1, 2));
split.add(scroll);
split.add(help);