George Town is the main point of interest in Penang, but it's very small and you can see it in as little as 2-3 days
Most of the attractions in George Town is in the UNESCO heritage historic center. You can walk around its entirety in a single afternoon.
With 2-3 days, not only can you cover the entire historic center, you also have time to check out surrounding areas like Gurney Drive, Penang Hill, Kek Lok Si, etc. which are all in the suburbs of George Town
However, personally I recommend spending 3-5 days in George Town. This will allow you to try all the different kinds of food available (Penang cuisine is a very good mix of Chinese, Malay, and Indian food because it's a multicultural place). Also, with this many days you can explore other parts of Penang outside of George Town, like Batu Ferringhi and Penang National Park