Diabetes mellitus is a well-known chronic metabolic disease that poses a long-term threat to human health and is characterized by a relative or absolute lack of insulin, resulting in hyperglycemia. Type 2 diabetes mellitus (T2DM) typically affects many metabolic pathways, resulting in β-cell dysfunction, insulin resistance, abnormal blood glucose levels, inflammatory processes, excessive oxidative reactions, and impaired lipid metabolism. It also leads to diabetes-related complications in many organ systems. Antidiabetic drugs have been approved for the treatment of hyperglycemia in T2DM; these are beneficial for glucose metabolism and promote weight loss, but have the risk of side effects, such as nausea or an upset stomach. A wide range of active components, derived from medicinal plants, such as alkaloids, flavonoids, polyphenol, quinones, and terpenoids may act as alternative sources of antidiabetic agents. They are usually attributed to improvements in pancreatic function by increasing insulin secretions or by reducing the intestinal absorption of glucose. Ease of availability, low cost, least undesirable side effects, and powerful pharmacological actions make plant-based preparations the key player of all available treatments. Based on the study of therapeutic reagents in the pathogenesis of humans, we use the appropriate animal models of T2DM to evaluate medicinal plant treatments. Many of the rat models have characteristics similar to those in humans and have the advantages of ease of genetic manipulation, a short breeding span, and access to physiological and invasive testing. In this review, we summarize the pathophysiological status of T2DM rat models and focus on several bioactive compounds from herbal medicine with different functional groups that exhibit therapeutic potential in the T2DM rat models, in turn, may guide future approach in treating diabetes with natural drugs.