This should be a comment (as it's not an answer) but I need some formatting and a lot of space here. The short version is that you need to find out why git index-pack is misbehaving or failing. (Fetching over a smart protocol normally retrieves a so-called thin pack, which git fetch needs to "fatten" using git index-pack --fix-thin.)
The "invalid index-pack output" error occurs if the output from git index-pack does not match what git fetch-pack expects. Here's the code involved:
char *index_pack_lockfile(int ip_out, int *is_well_formed)
{
char packname[GIT_MAX_HEXSZ + 6];
const int len = the_hash_algo->hexsz + 6;
/*
* The first thing we expect from index-pack's output
* is "pack\t%40s\n" or "keep\t%40s\n" (46 bytes) where
* %40s is the newly created pack SHA1 name. In the "keep"
* case, we need it to remove the corresponding .keep file
* later on. If we don't get that then tough luck with it.
*/
if (read_in_full(ip_out, packname, len) == len && packname[len-1] == '\n') {
const char *name;
if (is_well_formed)
*is_well_formed = 1;
packname[len-1] = 0;
if (skip_prefix(packname, "keep\t", &name))
return xstrfmt("%s/pack/pack-%s.keep",
get_object_directory(), name);
return NULL;
}
if (is_well_formed)
*is_well_formed = 0;
return NULL;
}
This is run from fetch-pack.c's get_pack function, which runs git index-pack with arguments based on a lot of variables. If you run your git clone with environment variable GIT_TRACE set to 1, you can observe Git running git index-pack. The call to index_pack_lockfile here only happens if do_keep is set, which is based on args->keep_pack initially, but can become set if the pack header's hdr_entries value equals or exceeds unpack_limit (see around line 859).
You can control the unpack_limit value using fetch.unpackLimit and/or transfer.unpackLimit. The default value is 100. You might be able to use these to work around some problem with index-pack, maybe—but index-pack should not be failing in whatever way it is failing. Note that if you want to force git fetch to use git unpack-objects instead, you must also disable object checking (fsck_objects).
It could be interesting to run git index-pack directly, too, on the data retrieved by git fetch. (Consider installing a shell script in place of the normal git index-pack, where the script prints its arguments, then uses kill -STOP on its own process group, so that you can inspect the temporary files.)
