Stir together sugar, cocoa powder and cornstarch in a medium saucepan until thoroughly mixed. Slowly whisk in milk.
Cook until thick over medium heat, stirring regularly, especially as mixture gets hot. Stir in vanilla. Pour into two small dishes and refrigerate at least 15 minutes to continue the thickening process.