Smartphones and tablet devices have been rapidly proliferating, and multi-touch interaction, powerful processors and rich array of sensors make these devices an attractive service platform for older users. While there is an increasing number of work investigating the issues that elderly users experience through their interaction with mobile devices, most have focused either on evaluation of low-level interaction characteristics or on qualitative survey. Therefore, we conducted a user study with 21 elderly participants to analyze the needs and issues faced by this user group under naturalistic usage scenarios. Specifically, we interviewed each participant about their experiences, had them perform various practical tasks using our custom testing application, and analyzed the operation logs using our custom visualizations. Based on our results, we summarize the types of issues observed, present design considerations for the applications studied, and future research directions.