Make the spice infusion: Add the water, cinnamon sticks, crushed cardamom pods, cloves, ginger, and black pepper to a medium saucepan. Bring to a gentle boil over medium-high heat.

Simmer the spices: Reduce the heat to medium-low and simmer uncovered for 5 minutes. The water should become fragrant and slightly darker as the spices infuse.

Brew the tea: Remove the saucepan from the heat and add the black tea bags. Steep for 4 minutes. Avoid steeping much longer, as the tea can become noticeably bitter.

Sweeten the chai: Remove the tea bags: Strain the chai through a fine-mesh strainer into a heatproof measuring cup or bowl. Stir in the brown sugar and vanilla while the tea is still hot. The sugar should dissolve completely.

Cool the concentrate: Let the chai cool at room temperature for about 10 minutes, then refrigerate it for about 20 minutes, or until thoroughly chilled. Do not pour hot chai directly over the ice because it will melt quickly and dilute the drink.

Build the lattes: Divide the ice between two 12-ounce (355 mL) glasses. Pour 1 cup (240 mL) of the chilled chai evenly between the glasses.

Add the milk: Pour 1/2 cup (120 mL) cold milk into each glass. Add 1 tablespoon (15 mL) heavy cream to each glass if using.

Stir and serve: Stir each latte until the chai and milk are evenly combined. Taste and add a little extra sweetener if desired. Serve immediately while cold.