The microhydrodynamics of particle suspensions in polymeric fluids has a wide range of applications in industry and biology. To discern the dynamics of particles in such systems, it is important to analyze the stress response of the suspension to applied flow fields. While such investigations have been theoretically done for suspensions of rigid spheres in weakly viscoelastic fluids, the effect of nonsphericity of particles on the stress remains relatively unexplored. The interplay between the response of the polymeric fluid and the particle orientation yields rich physics. The viscoelastic torques make the particle inhabit a preferred orientation in a given flow, resulting in time-dependent stresses. In this paper, we determine the average extra stress in a dilute suspension of rigid, non-Brownian spheroids in a second-order fluid subject to shear and extensional flows. We perform this task by examining the flow around a single spheroid in the limit of small Weissenberg number (W-i << 1) and perform an ensemble average of the stress tensor over all particle configurations. There are two contributions to the extra stress: one from the force dipole on the particles (stresslet) and another from the fluctuations in the velocity in the bulk fluid (fluid-induced particle stress), the latter of which does not arise in a zero Reynolds number Newtonian fluid. We present results for the O(phi Wi) corrections to the long-time effective shear viscosity, normal stress coefficients, and extensional viscosities in the suspension in shear, uniaxial extensional, and planar extensional flows, where phi is the particle volume fraction. To elucidate the effect of particle shape on the effective viscosity, we repeat this analysis for different aspect ratios (A(R)) for prolate (needlelike) and oblate (disklike) spheroids.