Autonomous vehicles (AVs)—vehicles that operate without real-time human input—are a potentially disruptive technology. If widely adopted, there is the potential for significant impacts on the energy and environmental characteristics of the transportation sector. This paper provides an outline of key drivers likely to influence the magnitude and direction of these impacts. We identify three broad categories: vehicle characteristics, transportation network, and consumer choice. Optimistically, AVs could facilitate unprecedented levels of efficiency and radically reduce transportation sector energy and environmental impacts; on the other hand, consumer choices could result in a net increase in energy consumption and environmental impacts. As the technology matures and approaches market penetration, improved models of AV usage, especially consumer preferences, will facilitate the development of policies that promote reductions in energy consumption.