The Barisan Nasional (BN) claimed control of the state government after the three elected representatives, two of whom are facing corruption charges, quit their parties and backed the ruling party as independents.
The Pakatan Rakyat (PR), or People's Alliance, in a last-ditch attempt to hold on to power, refused to cede control of the state despite orders by the sultan to hand power to the BN.
The political crisis has sparked public outrage and widespread allegations that the defectors had been bought over by Umno, the main partner in the ruling BN.
The BN was denied the two-thirds majority for the first time in 40 years after the PR, a loose opposition coalition, made unprecedented gains in the March 8 general election.
Perak's sultan, or Malay ruler, had a day earlier consented to the appointment of a new BN government, the coalition that has run Malaysia for 51 years, but the opposition remained defiant and threatened legal action.
The Perak imbroglio is expected to improve the chances of Najib Abdul Razak, the prime minister-in-waiting, in the run-up to party elections next month.
The opposition alliance said it would mount a legal challenge to the decision by the ruler of Perak's decision to recognise the new state government.
Mohd Nizar Jamaluddin, who became the chief minister after the general election, was escorted from his office on Friday but insisted: "I will go about my job as usual."
He said he would only quit if he is removed by a vote of no-confidence in the state assembly.
"As far as we are concerned, this is the legal government," Jamaluddin said before the swearing-in of the new BN chief minister.
Later in the afternoon police stopped more than 1,000 opposition supporters from marching to the Iskandariah Palace, the venue of the swearing in of Zambry Abdul Kadir as Perak's new BN chief minister.
There were reports that riot police fired tear gas at opposition supporters protesting against the seizure of state power.
Local media reported that at least 10 people were arrested.
Perak was one of five states won by the three-party opposition alliance led by Anwar Ibrahim, the country's former deputy prime minister.
Anwar, who last year tried to seize power nationally with the aid of defecting government MPs, said the opposition alliance would not quit the Perak government but would appeal to the royal leader for fresh polls.
"Such a takeover [in Perak] is really a big blow to democracy and insults the wisdom of the masses," he told The Associated Press news, while calling for fresh elections.