In addition to what the others have said, Postgres does not have a concept of a 'Clustered Index' like Microsoft SQL Server and other databases have. You can cluster an index, but it is a one-time operation (until you call it again) and will not maintain the clustering of rows upon edits, etc. See the docs
I was running into the same thing you were, where I half expected the rows to be returned in order of primary key (I didn't insert them out of order like you did, though). They did come back upon initial insert, but editing a record in Postgres seems to move the record to the end of the page, and the records quickly became out of order (I updated fields other than the primary key).