Aerodynamic design of projectiles is crucial to ensure that projectiles have the best performance during their flight. Performing aerodynamic design boils down to determining the optimal values of certain design variables of the projectile as the solution of a nonlinear optimization problem involving the stability derivatives of the projectile. Solving such an optimization problem involves a heavy procedure since either costly experimental tests or computationally intensive simulations are needed to obtain the stability derivatives for different values of the design variables. In this paper, a (cost-effective) Neural Network surrogate model is used to model the stability derivatives. A procedure balancing exploitation and exploration is then devised to determine, based on that surrogate model, the values of the design variables for which the stability derivatives have to be evaluated to both improve the surrogate model and approach the optimal design of the projectile. This framework is applied to optimize the geometrical configuration of a rectangular finner for a classical flight scenario.