Cohomology and extensions of representations of groups with normal Engel subgroups

Let λ , U be representations of a group G with a normal subgroup N . We study the first cohomology group under various spectral type conditions imposed on the restrictions of λ , U to N . We assume often that N is an Engel group and examine various decompositions of the extension e ( λ , U , ξ ) of λ by U associated with non-trivial ( λ , U ) -cocycles ξ . Aiming at applications to double extensions and the theory of J -unitary group representations on indefinite metric spaces, we describe ( λ , U ) -cocycles when G = D ⋉ N is the semidirect product, D is an Engel group and the restrictions of λ , U to N are χ 1 for some character χ on N (such pairs of representations form in a sense a base class in the variety of all pairs of representations). Our description is complete if , where is the group of all diagonal matrices with positive entries, or is the group of all upper triangular matrices d = ( d ij ) with d ii > 0 and T is its subgroup of all matrices with d ii = 1 , or G = SO ( 2 ) ⋉ R 2 .