The Tokugawa did not eventually collapse simply because of intrinsic failures. Foreign intrusions helped to precipitate a complex political struggle between the bakufu and a coalition of its critics. The continuity of the anti-bakufu movement in the mid-nineteenth century would finally bring down the Tokugawa. From the outset, the Tokugawa attempted to restrict families' accumulation of wealth and fostered a "back to the soil" policy, in which the farmer, the ultimate producer, was the ideal person in society. Despite these efforts to restrict wealth, and partly because of the extraordinary period of peace, the standard of living for urban and rural dwellers alike grew significantly during the Tokugawa period. Better means of crop production, transportation, housing, food, and entertainment were all available, as was more leisure time, at least for urban dwellers. The literacy rate was high for a preindustrial society, and cultural values were redefined and widely imparted throughout the samurai and chonin classes. Despite the reappearance of guilds, economic activities went well beyond the restrictive nature of the guilds, and commerce spread and a money economy developed. Although government heavily restricted the merchants and viewed them as unproductive and usurious members of society, the samurai, who gradually became separated from their rural ties, depended greatly on the merchants and artisans for consumer goods, artistic interests, and loans. In this way, a subtle subversion of the warrior class by the chonin took place.

A struggle arose in the face of political limitations that the shogun imposed on the entrepreneurial class. The government ideal of an agrarian society failed to square with the reality of commercial distribution. A huge government bureaucracy had evolved, which now stagnated because of its discrepancy with a new and evolving social order. Compounding the situation, the population increased significantly during the first half of the Tokugawa period. Although the magnitude and growth rates are uncertain, there were at least 26 million commoners and about 4 million members of samurai families and their attendants when the first nationwide census was taken in 1721. Drought, followed by crop shortages and starvation, resulted in twenty great famines between 1675 and 1837. Peasant unrest grew, and by the late eighteenth century, mass protests over taxes and food shortages had become commonplace. Newly landless families became tenant farmers, while the displaced rural poor moved into the cities. As the fortunes of previously well-to-do families declined, others moved in to accumulate land, and a new, wealthy farming class emerged. Those people who benefited were able to diversify production and to hire laborers, while others were left discontented. Many samurai fell on hard times and were forced into handicraft production and wage jobs for merchants.

Western intrusions were on the increase in the early nineteenth century. Russian warships and traders encroached on Karafuto (called Sakhalin under Russian and Soviet control) and on the Kuril Islands, the southernmost of which are considered by the Japanese as the northern islands of Hokkaido. A British warship entered Nagasaki Harbor searching for enemy Dutch ships in 1808, and other warships and whalers were seen in Japanese waters with increasing frequency in the 1810s and 1820s. Whalers and trading ships from the United States also arrived on Japan's shores. Although the Japanese made some minor concessions and allowed some landings, they largely attempted to keep all foreigners out, sometimes using force. Rangaku became crucial not only in understanding the foreign "barbarians" but also in using the knowledge gained from the West to fend them off.

By the 1830s, there was a general sense of crisis. Famines and natural disasters hit hard, and unrest led to a peasant uprising against officials and merchants in Osaka in 1837. Although it lasted only a day, the uprising made a dramatic impression. Remedies came in the form of traditional solutions that sought to reform moral decay rather than address institutional problems. The shogun's advisers pushed for a return to the martial spirit, more restrictions on foreign trade and contacts, suppression of Rangaku, censorship of literature, and elimination of "luxury" in the government and samurai class. Others sought the overthrow of the Tokugawa and espoused the political doctrine of sonno-joi (revere the emperor, expel the barbarians), which called for unity under imperial rule and opposed foreign intrusions. The bakufu persevered for the time being amidst growing concerns over Western successes in establishing colonial enclaves in China following the Opium War of 1839-42. More reforms were ordered, especially in the economic sector, to strengthen Japan against the Western threat.

Japan turned down a demand from the United States, which was greatly expanding its own presence in the Asia-Pacific region, to establish diplomatic relations when Commodore James Biddle appeared in Edo Bay with two warships in July 1846. However, when Commodore Matthew C. Perry's four-ship squadron appeared in Edo Bay in July 1853, the bakufu was thrown into turmoil. The chairman of the senior councillors, Abe Masahiro (1819-57), was responsible for dealing with the Americans. Having no precedent to manage this threat to national security, Abe tried to balance the desires of the senior councillors to compromise with the foreigners, of the emperor who wanted to keep the foreigners out, and of the daimyo who wanted to go to war. Lacking consensus, Abe decided to compromise by accepting Perry's demands for opening Japan to foreign trade while also making military preparations. In March 1854, the Treaty of Peace and Amity (or Treaty of Kanagawa) opened two ports to American ships seeking provisions, guaranteed good treatment to shipwrecked American sailors, and allowed a United States consul to take up residence in Shimoda, a seaport on the Izu Peninsula, southwest of Edo. A commercial treaty, opening still more areas to American trade, was forced on the bakufu five years later.

The resulting damage to the bakufu was significant. Debate over government policy was unusual and had engendered public criticism of the bakufu. In the hope of enlisting the support of new allies, Abe, to the consternation of the fudai, had consulted with the shinpan and tozama daimyo, further undermining the already weakened bakufu. In the Ansei Reform (1854-56), Abe then tried to strengthen the regime by ordering Dutch warships and armaments from the Netherlands and building new port defenses. In 1855 a naval training school with Dutch instructors was set up at Nagasaki, and a Western-style military school was established at Edo; by the next year, the government was translating Western books. Opposition to Abe increased within fudai circles, which opposed opening bakufu councils to tozama daimyo, and he was replaced in 1855 as chairman of the senior councillors by Hotta Masayoshi (1810-64).

At the head of the dissident faction was Tokugawa Nariaki, who had long embraced a militant loyalty to the emperor along with antiforeign sentiments, and who had been put in charge of national defense in 1854. The Mito school--based on neo-Confucian and Shinto principles--had as its goal the restoration of the imperial institution, the turning back of the West, and the founding of a world empire under the divine Yamato Dynasty.

In the final years of the Tokugawa, foreign contacts increased as more concessions were granted. The new treaty with the United States in 1859 allowed more ports to be opened to diplomatic representatives, unsupervised trade at four additional ports, and foreign residences in Osaka and Edo. It also embodied the concept of extraterritoriality (foreigners were subject to the laws of their own countries but not to Japanese law). Hotta lost the support of key daimyo, and when Tokugawa Nariaki opposed the new treaty, Hotta sought imperial sanction. The court officials, perceiving the weakness of the bakufu, rejected Hotta's request and thus suddenly embroiled Kyoto and the emperor in Japan's internal politics for the first time in many centuries. When the shogun died without an heir, Nariaki appealed to the court for support of his own son, Tokugawa Yoshinobu (or Keiki), for shogun, a candidate favored by the shinpan and tozama daimyo. The fudai won the power struggle, however, installing Tokugawa Yoshitomi, arresting Nariaki and Keiki, executing Yoshida Shoin (1830-59, a leading sonno-joi intellectual who had opposed the American treaty and plotted a revolution against the bakufu), and signing treaties with the United States and five other nations, thus ending more than 200 years of exclusion.

The strong measures the bakufu took to reassert its dominance were not enough. Revering the emperor as a symbol of unity, extremists wrought violence and death against the bakufu and han authorities and foreigners. Foreign naval retaliation led to still another concessionary commercial treaty in 1865, but Yoshitomi was unable to enforce the Western treaties. A bakufu army was defeated when it was sent to crush dissent in Satsuma and Choshu han in 1866. Finally, in 1867, the emperor died and was succeeded by his minor son Mutsuhito; Keiki reluctantly became head of the Tokugawa house and shogun. He tried to reorganize the government under the emperor while preserving the shogun's leadership role. Fearing the growing power of the Satsuma and Choshu daimyo, other daimyo called for returning the shogun's political power to the emperor and a council of daimyo chaired by the former Tokugawa shogun. Keiki accepted the plan in late 1867 and resigned, announcing an "imperial restoration." The Satsuma, Choshu, and other han leaders and radical courtiers, however, rebelled, seized the imperial palace, and announced their own restoration on January 3, 1868. The bakufu was abolished, Keiki was reduced to the ranks of the common daimyo, and the Tokugawa army gave up without a fight (although other Tokugawa forces fought until November 1868, and bakufu naval forces continued to hold out for another six months).

Data as of January 1994