How can we design agents able to autonomously explore and perform (scientific) discovery? In my research, I tackle this question from the perspective of sequential decision-making by working on foundations and practical algorithms at the intersection of reinforcement learning, bayesian optimization, experimental design, and submodular optimization.