Physics-Informed Neural Networks (PINNs) offer an efficient approach to solving partial differential equations (PDEs). In theory, they can provide the solution to a PDE at an arbitrary point for the computational cost of a single forward pass of a neural network.However, PINNs often pose challenges during training, necessitating complex hyperparameter tuning, particularly for PDEs with oscillating solutions.In this paper, we propose a PINN training scheme for PDEs with oscillating solutions. We analyze the impact of sinusoidal activation functions as model prior and incorporate self-adaptive weights into the training process.Our experiments utilize the double mass-spring-damper system to examine shortcomings in training PINNs.Our results show that strong model priors, such as sinusoidal activation functions, are immensely beneficial and, combined with self-adaptive training, significantly improve performance and convergence of PINNs.