First Lets Talk About 1.5V battery
Batteries form a cylindrical shape to put the chemical electrolyte paste in a layer around the center electrode. This is the most efficient way of doing that. If you roll a bandage over your hand it will be in cylindrical shape and the bandage is uniform around your hand. So I think you understood why the batteries are cylindrical.
Now you may be confused why the 9V batteries are rectangular right. That’s normal.
I think you know when batteries are connected in series (one after another, like (+ -)(+ -)(+ -) ) the voltage gets added. That means if the three batteries are 1.5 volts each the net voltage will be (1.5 + 1.5 + 1.5) =4.5V. But if you put the batteries in parallel (all battery’s same terminals are connected ) then you will get 1.5V.
So if you want to make 9V using 1.5V batteries you need six 1.5V batteries (6*1.5= 9V) connected in series.
If you open a 9V battery carefully, you will see six batteries. All of which are 1.5V and in a cylindrical shape. Here are few pictures.
So I think it is clear to you that why they are in rectangular shape.