Objective To prepare human SET8 monoclonal antibody and explore its effects on the proliferation, apoptosis and cell cycle of hepatoma cells, and evaluate its anti-tumor effect in mouse models of hepatocellular carcinoma. Methods By immunizing mice with human SET8 polypeptide fragment, B cells and myeloma cells were screened and fused to establish hybridoma cell line that secreted SET8 monoclonal antibody stably. The production was expanded by intraperitoneal injection of mice and collection and purification of ascites. The CCK-8, flow cytometry and Western blot were used to analyze the effects of SET8 monoclonal antibody on the proliferation, apoptosis, cell cycle and apoptosis-related protein expression of hepatocellular carcinoma cells. Finally, a mouse model of human hepatocellular carcinoma cell transplantation was constructed to evaluate the inhibitory effect of SET8 monoclonal antibody on tumor growth in vivo. Results Human SET8 monoclonal antibody significantly inhibited the viability of Huh-7 and Mahlavu hepatoma cells at the concentration of 50ug/ml and 100ug/ml, and the higher the concentration, the stronger the inhibitory effect. Flow cytometry analysis showed that compared with the control group, SET8 monoclonal antibody, paclitaxel and their combination treatment significantly increased the apoptosis rate of Mahlavu cells, and the combination group had the best effect. Meanwhile, SET8 monoclonal antibody induced Mahlavu cell cycle arrest in S and G2 phases and reduced G1 phase cells. Western blot analysis showed that the monoclonal antibody increased the expression of apoptosis-related proteins Bax and caspase-3. Further animal experiments showed that SET8 monoclonal antibody alone or in combination with paclitaxel effectively inhibited the proliferation of hepatocellular carcinoma cells in nude mice, and the combination therapy had the most significant effect. Conclusion The prepared human SET8 monoclonal antibody effectively inhibited the proliferation of hepatocellular carcinoma cells, promoted the apoptosis of hepatocellular carcinoma cells, and showed a good anti-tumor effect in animals.