The idea to ditch the keyboard for what Jobs calls a multi-touch display came about in the early 2000s, although the company was working on a telephone at the time, he said.
That's when a prototype was brought to him that used the device's now-famous scrolling mechanism.
"I thought, 'My God we can build a phone out of this,'" Jobs said at The Wall Street Journal's "D: All Things Digital" conference in Rancho Palos Verdes.
But the tablet product was put on the shelf, the iPhone went into development for several years before making its debut in 2007. Apple only started selling the iPad tablet computer in April.
Both products have taken on more of the personal computing tasks once handled by computers running Microsoft's Windows operating system and other programs.
That's helped Apple surpass Microsoft, its longtime nemesis, as the largest technology company in the world by market capitalisation - a milestone achieved last week.
"It's surreal," Jobs said.
Jobs was asked why the once close relationship between Google and Apple has lately turned tense. Google has developed a web browser and a computer operating system and began selling its own cell phone in January a bid to upstage the iPhone.
"They decided to compete with us, and so they are," Jobs said.
Although he acknowledged the popularity of the iPhone has caused network troubles at carrier AT&T, Jobs said the problems would have happened to any wireless phone company with that many iPhone and iPad users.
Ahead of a major software developers conference scheduled to begin June 7 in San Francisco, Jobs was characteristically tightlipped about what, if any, new features would be on new iPhone models.
As for the iPad's stunning popularity, Jobs said the device was beginning to erode the usefulness of the personal computer.
"We like to talk about the post-PC era," he said. "I think we're embarked on that. Is it the iPad? Who knows?
"Will it happen next year or five years from now or seven years? Who knows?"
