Castle walls were made from stone, usually granite. They were carried or hauled to the castle site to build. Masons would form the stone and workers would haul them into place. The bottom of the walls were sloped up so when an enemy attacked with rams they would be ineffective. The walls were also filled with rubble much like the Great Wall of China to save money on building.
Castlewalls were an evolutionary defense structure over a period of thousands of years.
Castlewalls are seen in all cultures, from Mesopotamian, to Arabian, and to (most famously) western European cultures.
Although castlewalls initially started out as a round circle of logs, but with the advent of stone wall construction, it was easier to build walls in neat, straight lines and 90° corners to create a box shape that was also appealing to the eye.