Introduction

In today’s fast-paced digital world, seamless and secure payment processing is critical for businesses. From August 2023 to June 2024, I led the development of an advanced Online Payment Gateway, aimed at integrating multiple payment providers through a unified API interface, ensuring PCI-DSS compliance. This blog post outlines the challenges, technologies, and solutions we implemented to deliver a scalable, secure, and flexible payment gateway capable of handling thousands of transactions daily.

Project Overview

The core objective was to develop a unified API solution that could support multiple payment providers. This required:

• PCI-DSS compliance for secure credit card processing.
• A flexible, adaptable system to handle different providers’ integration options and data models.
• A robust architecture capable of supporting high transaction volumes while maintaining system performance.

We utilized Nest.JS as the primary backend framework and implemented a microservice architecture to ensure scalability and reliability.

Key Features:

• Unified API Interface for multiple payment providers.
• Microservice Architecture with independent services for transaction handling.
• Advanced Error Handling and performance monitoring with Sentry, Grafana, and Prometheus.
• Frontend Integration with iframes and SDKs for seamless payment processing.

Challenges and Solutions

1. Standardizing Payment Providers

One of the biggest challenges was unifying the different integration methods and data models of various payment providers. To avoid code duplication and manage multiple providers efficiently, we employed the Adapter Pattern. This allowed us to handle each provider’s specific requirements while maintaining a unified system for managing API and frontend integrations.

For the dependency injection and easy management of new providers, we used Nest.JS, enabling us to inject adapters dynamically and keep the architecture clean and scalable.

2. Handling Different Payment Types

The system needed to support both bank transfers and credit card payments, which required distinct APIs due to their differing interfaces and PCI-DSS compliance requirements. Since we weren't fully PCI-DSS certified at the time, handling sensitive data such as credit card PAN (Primary Account Number) was not an option. Instead, we integrated a tokenization process that would allow secure transmission of payment data without storing it.

We built SDKs to manage this process, dynamically tokenizing payment data and synchronizing it with our backend without the need for major updates when onboarding new payment providers.

3. Frontend Integration

Many clients requested frontend integration through iframes or JavaScript SDKs. To ensure flexibility, we implemented an SDK with low-level functions like init and pay, dynamically processed by each provider’s script. We used this approach to create Hosted Checkout solutions, offering a complete checkout process via iframe or redirect options.

The frontend was developed with Pug and Tailwind CSS, and we leveraged Nest.JS for server-side rendering, ensuring responsive design and fast load times. This resulted in a highly performant, SEO-optimized frontend that catered to both desktop and mobile users.

4. Performance and Scalability

To ensure the system could handle thousands of transactions daily, we implemented a microservice architecture. Each service, from transaction management to messaging, had its own database, avoiding bottlenecks and improving overall performance. These services communicated through Redis for fast data transmission and synchronization.

We provided extensive reporting capabilities via Metabase, embedded into the UI. The backend services were supported by a separate MongoDB instance to guarantee optimal performance.

5. Error Handling and Monitoring

Given the critical nature of the application, real-time monitoring and error tracking were essential. We integrated Sentry for detailed error logging and utilized Grafana Loki for logging, Tempo for tracing, and Prometheus for general metrics and alerts. This setup allowed us to trace every request and error across microservices, ensuring high system reliability.

We also connected our alerts to Slack, enabling quick responses to any performance or operational issues.

6. End-to-End Testing and CI/CD

To ensure the system's reliability, we implemented comprehensive end-to-end (E2E) tests for every API endpoint using Supertest and Jest. For frontend testing, we used Cypress through the nx framework. The tests were run periodically in a containerized environment, pushing notifications and reports to Slack when necessary.

We also built a CI/CD pipeline that deployed backend services and the frontend automatically on the client's servers, ensuring continuous delivery of updates.

API Documentation and User Testing

A key project requirement was the creation of a detailed and flexible API documentation. We chose Docusaurus after testing various solutions, which provided excellent documentation capabilities and flexibility. We combined:

• Manually crafted descriptions of API endpoints.
• React-based components for integration testing.
• Auto-generated API specifications using OpenAPI/Swagger.

The documentation included diagrams generated from our Stately state machines, offering a clear representation of payment states. To facilitate user testing, we integrated Keycloak into the test system, allowing clients to interact with the API directly from the documentation.

Conclusion

Building the Online Payment Gateway was an exciting challenge that involved complex integrations, stringent security requirements, and a focus on performance and scalability. Through the use of microservices, Nest.JS, and advanced tools like Sentry and Grafana, we successfully delivered a robust, flexible, and PCI-DSS compliant payment system.

The system is now capable of handling thousands of transactions daily, provides seamless frontend and backend integrations, and is fully scalable for future growth. It serves as a strong foundation for future payment-related projects, offering businesses a secure and efficient way to process payments.