The air force is down to 33 squadrons, against its requirement of 45 to face both China, with which it has a festering border dispute, and nuclear-armed rival Pakistan.
French Defence Minister Jean-Yves Le Drian signed the agreement with his Indian counterpart, Manohar Parrikar, in New Delhi, ending almost 18 months of wrangling over terms between New Delhi and manufacturer Dassault Aviation.
India's defence ministry said it would confirm the exact price later on Friday, but a ministry official said it was 7.8 billion euros ($8.7 billion).
Air force officials have warned for years about a major capability gap opening up with China and Pakistan without new state-of-the-art planes, as India's outdated and largely Russian-made fleet retires and production of a locally made plane was delayed.
India had originally awarded Dassault with an order for 126 Rafales in 2012 after the twin-engine fourth-generation fighter beat rivals in a decade-long selection process, but subsequent talks collapsed.
Modi, who has vowed to modernise India's armed forces with a $150 billion spending spree, personally intervened in April 2015 to agree on the smaller order of 36 and give the air force a near-term boost as he weighed options for a more fundamental overhaul.
The first ready-to-fly Rafales are expected to arrive by 2019 and India is set to have all 36 within six years.