Groups without Faithful Transitive Permutation Representations of Small Degree

A subgroupHof a groupGiscore-freeifHcontains no non-trivial normal subgroup ofG, or equivalently the transitive permutation representation ofGon the cosets ofHis faithful. We study the obstacles to a group having large core-free subgroups. We call a subgroupDa “dedekind” subgroup ofGif all subgroups ofDare normal inG. Our main result is the following: If a finite groupGhas no core-free subgroups of order greater thank, thenGhas two dedekind subgroupsD1andD2such that every subgroup inGof order greater thanf(k) has non-trivial intersection with eitherD1orD2(wherefis a fixed function independent ofG). Examples show that the dedekind subgroups need not have index bounded by a function ofk, and the result would not be true with one dedekind subgroup instead of two. We exhibit various related properties ofp-groups and infinite locally finite groups without large core-free subgroups, including the following: IfGis a locally finite group with no infinite core-free subgroup, then every infinite subgroup ofGcontains a non-trivial cyclic normal subgroup ofG. We also exhibit asymptotic bounds for some related problems, including the following: If a groupGhas a solvable subgroup of indexn, thenGhas a solvable normal subgroup of index at mostnc(for some absolute constantc). IfGis a transitive permutation group of degreenwith cyclic point-stabilizer subgroup, then |G|