Geometrically induced birefringence represents a pathway for precisely engineering the modes in fibers and is particularly relevant for applications that crucially depend on modal dispersion. Here liquid core fibers (LCFs) with elliptical cores are analyzed in view of modal properties and third-harmonic generation (THG) numerically and experimentally. Using finite element modeling, the impact of ellipticity on phase matching, inter-modal coupling, electric field distribution, and birefringence are investigated. Significant THG in practically relevant modes, in accordance with phase-matching calculations, was measured in inorganic solvent-based LCFs.