ListView with multiple Adapters and SectionIndexers

Question

I have an activity that contains a ListView that can be populated by different adapters. Every adapter that may populate the list is an instance of a specific class, and every one of those classes implements the SectionIndexer interface. I have enabled FastScroll in my listview, and it works fine the first time I use ListView.setAdapter(Adapter X). However, when I use ListView.setAdapter(Adapter Y), It still uses the previous adapter's SectionIndexer (Adapter X).
How can I "reset" the ListView's SectionIndexer?