For dough: In the bowl of a stand mixer, whisk together flour, granulated sugar, yeast, lemon zest, and salt by hand. Add warm milk; using the paddle attachment, beat at low speed until combined. Add egg and vanilla, beating just until dough comes together and stopping to scrape sides of bowl.
Switch to the dough hook attachment. With mixer on low speed, add butter, one cube at a time, beating until combined after each addition. Beat until dough is elastic and smooth, 10 to 12 minutes.
Oil a large bowl. Place dough in bowl, turning to grease top. Cover and let rise in a warm, draft-free place (75°F/24°C) until doubled in size, about 1 hour. (Alternatively, cover and refrigerate overnight; let dough come to room temperature before continuing with recipe.)
Lightly punch down dough. Cover and let stand for 10 to 15 minutes.
Line a 13×9-inch rimmed baking sheet with parchment paper. Turn out dough onto a lightly floured surface, and roll into a 13×9-inch rectangle. Transfer dough to prepared pan; let stand at room temperature for 20 to 30 minutes.
For topping: In a small bowl, whisk together flour, 2 tablespoons (24 grams) granulated sugar, brown sugar, almonds, and salt until well combined. Stir in melted butter and vanilla until mixture is well combined and crumbly.
In a medium bowl, stir together plums, cinnamon, and remaining 3 tablespoons (36 grams) granulated sugar. Place plums in a single layer on dough, overlapping as needed. Drizzle any juices onto plums. Sprinkle almond mixture all over plums. Cover with plastic wrap, and let stand until puffed, about 30 minutes.
Preheat oven to 350°F (180°C).
Bake until edges are golden brown and an instant-read thermometer inserted in center registers 190°F (88°C), 20 to 25 minutes. Let cool on a wire rack for 20 minutes; serve warm. Store in an airtight container for up to 3 days.