How to Synchronize Changes from a Source Repo to a Destination Repo Triggered from a Push to the Source Main Except Some Files

Are you tired of manually updating your destination repository every time you make changes to your source repository? Do you want to automate the process and ensure that your changes are reflected in real-time, except for a few files that you want to keep separate? Look no further! In this article, we’ll show you how to synchronize changes from a source repo to a destination repo triggered from a push to the source main, while excluding specific files.

Why Do You Need to Synchronize Changes Between Repos?

In today’s fast-paced software development world, it’s common to have multiple repositories that serve different purposes. You might have a source repository where you make changes to your code, and a destination repository where you deploy your application. Keeping these repositories in sync is crucial to ensure that your changes are reflected in the production environment. Manual synchronization can be time-consuming and prone to errors, which is why automating the process is essential.

Prerequisites

Before we dive into the tutorial, make sure you have the following:

• A source repository with a main branch (e.g., GitHub, GitLab, or Bitbucket)
• A destination repository with a main branch (e.g., GitHub, GitLab, or Bitbucket)
• Access to both repositories with push and pull permissions
• A GitHub Actions or GitLab CI/CD account set up (we’ll use GitHub Actions in this example)

Step 1: Create a GitHub Actions Workflow File

Log in to your GitHub account and navigate to your source repository. Create a new file in the `.github/workflows` directory called `sync-changes.yml`. This file will contain the workflow script that will trigger the synchronization process.

  
name: Sync Changes to Destination Repo

on:
  push:
    branches:
      - main

jobs:
  sync:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v2
      - name: Sync changes to destination repo
        env:
          DEST_REPO_USERNAME: ${{ secrets.DEST_REPO_USERNAME }}
          DEST_REPO_TOKEN: ${{ secrets.DEST_REPO_TOKEN }}
          DEST_REPO_URL: ${{ secrets.DEST_REPO_URL }}
        run: |
          git clone ${DEST_REPO_URL} dest-repo
          git checkout -b main
          git pull origin main
          git checkout origin/main
          git reset --hard HEAD
          git add .
          git commit -m "Sync changes from source repo"
          git push -u origin main
  

Let’s break down the script:

• The workflow is triggered on push events to the main branch of the source repository.
• The job runs on an ubuntu-latest environment.
• The first step checks out the code in the source repository.
• The second step syncs the changes to the destination repository using the stored secrets.

Step 2: Store Secrets in GitHub Actions

To store the destination repository credentials and URL as secrets, follow these steps:

1. Navigate to your repository’s settings by clicking on the gear icon on the top-right corner.
2. Click on “Actions” in the left-hand sidebar.
3. Click on “Secrets” in the left-hand sidebar.
4. Click on “New repository secret” and add the following secrets:

Secret Name	Description	Value
DEST_REPO_USERNAME	Destination repository username	your-destination-repo-username
DEST_REPO_TOKEN	Destination repository personal access token	your-destination-repo-token
DEST_REPO_URL	Destination repository URL	https://github.com/your-destination-repo-owner/your-destination-repo-name.git

Step 3: Exclude Specific Files from the Synchronization Process

To exclude specific files from the synchronization process, you’ll need to modify the GitHub Actions workflow file. Add the following code before the `git add .` command:

  
      - name: Exclude files from synchronization
        run: |
          git rm --cached $(git ls-files --exclude-from=.gitignore --ignored --exclude='*.txt')
  

This code uses `git ls-files` to list all files that are ignored by Git (using the `.gitignore` file) and excludes files with the `.txt` extension. You can modify the command to exclude files based on your specific requirements.

Step 4: Test the Workflow

Make some changes to your source repository, commit them, and push them to the main branch. The GitHub Actions workflow should trigger automatically and synchronize the changes to the destination repository, excluding the specified files.

Conclusion

In this article, we showed you how to synchronize changes from a source repo to a destination repo triggered from a push to the source main, while excluding specific files. By automating the synchronization process, you can ensure that your changes are reflected in real-time, without manual intervention. This workflow can be adapted to fit your specific requirements, so feel free to get creative and modify the script to suit your needs!

Remember to keep your secrets secure and never hardcode them in your workflow file. Use the built-in secrets feature provided by GitHub Actions or your CI/CD tool of choice.

Additional Tips and Variations

If you want to synchronize changes in both directions (i.e., from source to destination and vice versa), you can create two separate workflows, one for each direction.

You can also modify the workflow to synchronize changes only for specific files or directories by modifying the `git add` and `git rm` commands.

If you’re using a different CI/CD tool, such as GitLab CI/CD or CircleCI, you can adapt the workflow script to fit their syntax and features.

FAQs

Q: What if I have multiple source repositories that I want to synchronize with a single destination repository?

A: You can create multiple workflows, one for each source repository, and use the same destination repository URL in each workflow.

Q: Can I use this workflow for repositories hosted on different platforms, such as GitHub and GitLab?

A: Yes, you can use this workflow with repositories hosted on different platforms by modifying the repository URL and credentials accordingly.

Q: What if I want to synchronize changes only for specific branches, not just the main branch?

A: You can modify the `on` section of the workflow file to specify the branches that should trigger the synchronization process.

We hope this article has helped you automate the synchronization process between your source and destination repositories. Happy coding!

Frequently Asked Question

Get to know the secrets of synchronizing changes between repositories with ease!