Deriving from AdapterView can work, but it may not be as beneficial as you hope. Some of the infrastructure provided by AdapterView is package-private, meaning we don't have access to it.  
For example, AdapterView manages the selected item index for AbsListView and ListView. However, because methods like setNextSelectedPositionInt(int position) (which is the only path to setting mNextSelectedPosition) are package-private, we can't get to them. AbsListView and ListView can get to them because they're in the same package, but we can't.
(If you dig into the AdapterView source you'll find that setNextSelectedPositionInt() is called from handleDataChanged().  Unfortunately handleDataChanged() is also package-private and is _not_called from anywhere else within AdapterView that could be leveraged to enable setting position.)
That means that if you need to manage selected position, you'll need to recreate that infrastructure in your derived class (or you'll need to derive from ListView or AbsListView...though I suspect you'll run into similar problems deriving from AbsListView).  It also means that any AdapterView functionality that revolves around item selection likely won't be fully operational.