Mix the yeast, water, flour, milk, oregano, salt, and olive oil at medium speed in a mixer fixed with a bread hook until smooth and elastic. Put the dough in an oiled bowl and let rise until doubled in size, about 45 minutes.
Punch the dough down and let rest for five minutes. Gently knead the olives into the dough and let rest ten minutes more. Heat the oven to 375.
Punch the dough down again and divide it into sixteenths. Roll each piece into a ball, put the pieces on greased baking sheets, cover with plastic wrap, and let rise until doubled in size, about 30 minutes.
Cut an X into the top of each roll. Bake for 18 minutes; sprinkle with cheese as soon as the rolls come out of the oven.