This project focuses on developing an autonomous ultrasound scanning system using Unity, ML-Agents, and Soft Actor-Critic (SAC) reinforcement learning. I built a high-fidelity simulation environment with deformable tissue models (CRESSim-MPM) and synthetic ultrasound rendering, enabling agents to learn realistic probe–tissue interactions. A major step forward was transitioning from location-based to image-based rewards, allowing the agent to learn directly from ultrasound imagery rather than positional cues. Through curriculum learning, domain randomization, and integrated soft-body physics, I created a robust and clinically relevant simulation pipeline for AI-guided medical imaging.