Asymptotic Eigenfunctions of the Operator ∇D(x)∇ Defined in a Two-Dimensional Domain and Degenerating on Its Boundary and Billiards with Semi-Rigid Walls

We propose a method for constructing asymptotic eigenfunctions of the operator ̂L = ∇ D ( x 1 ,x 2 )∇ defined in a domain Ω ? R 2 with coefficient D ( x ) degenerating on the boundary ∂ Ω. Such operators arise, for example, in problems about long water waves trapped by coasts and islands. These eigenfunctions are associated with analogs of Liouville tori of integrable geodesic flows with the metric defined by the Hamiltonian system with Hamiltonian D ( x ) p 2 and degenerating on ∂ Ω. The situation is unusual compared, say, with the case of integrable two-dimensional billiards, because the momentum components of trajectories on such “tori” are infinite over the boundary, where D ( x ) = 0, although their projections onto the plane R 2 are compact sets, as a rule, diffeomorphic to annuli in R 2 . We refer to such systems as billiards with semi-rigid walls.