Japan’s ruling coalition is poised to lose its majority in the country’s upper house of parliament — the House of Councillors — according to projections after polls closed Sunday.

The NHK public broadcaster predicted it “may be difficult for the ruling coalition to maintain their majority” as initial results came in, with the ruling conservative Liberal Democratic Party (LDP) and its junior coalition partner, the Buddhist Komeito party, were forecast to lose seats. The LDP has governed Japan for most of the country’s modern history.

NHK also projected that the main opposition party, the liberal Constitutional Democratic Party (CDP), would “expand its seat count,” while the populist conservative Democratic Party for the People and the far-right Sanseito also registered “significant gains.”

Sanseito ran an anti-immigration campaign under a “Japanese First” slogan.

Prime Minister Shigeru Ishiba told a press conference that he was “aware of [his] responsibility,” but said he planned to stay on as head of government. Ishiba is in talks with Washington in an attempt to avoid a 25 percent U.S. tariff scheduled to take effect Aug. 1. U.S. President Donald Trump has criticized Japan for not importing more American rice despite a domestic shortage and for resisting calls to increase its spending on defense.

Sunday’s electoral setback comes just a few months after the ruling coalition lost control of the lower House of Representatives in October. The lower house is the more powerful of the two, as it selects the prime minister and can override the upper house on budgets and legislation.

While Ishiba has only been in office since Oct. 1 last year, past leaders have struggled to remain in power after losing control of the upper house.

“Clearly, people have spoken — people have said no to Ishiba’s administration,” former Prime Minister and CDP leader Yoshihiko Noda said Sunday, adding he would wait for Ishiba to explain the conditions under which he intended to remain in his current role.