Abstract: Feature fusion is a powerful technique that enables predictors to access a semantically rich representation of an image. Feature Pyramid Networks (FPNs) are the most widely used models for fusing features. However, the context within the FPN layers is inconsistent, leading to false predictions. This article addresses the context inconsistency in FPN and proposes CMFPN, a new design that improves feature fusion by decoupling feature aggregation from context modeling. Experimental results, based on the COCO dataset, show that CMFPN effectively resolves the context issues and enhances the Average Precision (AP) results for both object detection and instance segmentation by $2.30\%$ and $1.7\%$, respectively.