Integration test

We have an integration test that does a time-based stress test of the team feature: it spins up multiple threads that check out a single team project and concurrently commit changes to it.

See /test-integration/src/org/omegat/core/data/TestTeamIntegration.java for details.

An integration test is test for team project concurrent modification. It doesn’t simple junit test, but looks like ‘integration’ test.

This test prepares a scenario, executes separate JVMs for concurrent updates, then check remote repository data.

All child process updates own segments with source1…5/0/1/2/3 by values from 1 and more. Segment source/0 updated each time, but source/1/2/3 updated once per cycle. After the process will be finished, values in tmx should be in the right order, i.e. only by increasing order. That means user will not commit previous translation for another user’s segments.

All users will modify segment with ‘concurrent’ a source by values from 1 and more with user’s prefix. Conflicts should be resolved by choosing higher value. After processes will be finished, values in ‘concurrent’ segment should be also increased only.

Each child saves {@code Integer.MAXVALUE} as last translation, and it is not mandatory to be committed by the current OmegaT implementation. See “GIT_CONFLICT=Push failed, Will be synchronized next time.”

Note that when using a git repository accessed by the file:// protocol, gc can cause concurrency issues that result in an error saving the project and missed sync cycles. When this happens near the end of the test will fail (seen with JGit 4.8).

This has not been seen when using git+ssh protocol. This is unlikely to be a problem in real-world scenarios, but a workaround for this test is to disable gc on the repo with

git config gc.auto 0
git config gc.autodetach false
git config receive.autogc false

Integration Test with container tools

Docker

Docker and docker compose are popular to prepare a pragmatic test environment. You can find docker configurations Dockerfile in test-integration/docker/

A test setup requires two containers.

  1. server: ssh+git, git https, http content, http-dav and ssh+svn server container to provide team repository and mapping

  2. client: omegat test instance container to run integration test.

OmegaT project provide compose.yml configuration that orchestrate test setup and execution in one line command.

The Server has FOUR individual contents.

  1. Git team: ssh+git and git https protocol provide omegat team repository

  2. SVN team 1: http-dav provide omegat team repository without authentication on http://server/svn/omegat-test.svn

  3. SVN team 2: ssh+svn omegat team repository on /home/git/omegat-test.svn

  4. Plain HTTP server: http://server/README provide content for mapping

Repositories are independent among each other.

install Compose V2

Compose V1 is deprecated in June 2023 You are recommend to use Compose V2 which act as a docker plugin. When you install a Docker Desktop, you may already have a Docker Compose V2.

When you install a docker command from Linux distribution, we need to install compose V2 manually. Please visit a compose manual page for instructions.

containerd, runc, buildkit and nerdctl

There are alternative toolsets to support container operations. Open Container Initiative creates open standard around container format and runtimes.

OCI runtime spec v1.1 defines the behavior and the configuration interface of low-level container runtime such as runc. These container runtimes are called from high-level container runtimes such as containerd.

nerdctl is a docker compatible CLI for containerd that also support compose subcommand.

How to run with docker command

WARNING: You should operate all commands from project root directory.

Before you start test, you need to build test containers.

docker compose -f compose.yml build

then you can run with default settings

docker compose -f compose.yml up

You can stop integration test processes by pressing Ctrl-C key or command.

docker compose -f compose.yml down

We can omit -f option because a default script filename is compose.yml

docker compose up
docker compose down

The Default duration of execution is 4200 second (= 40 min.) When you want to specify duration, you can set environment variable and give the docker-compose option to stop servers after test finished.

env DURATION=600 docker compose -f compose.yml up \
    --abort-on-container-exit 
docker compose down

This case specifies duration as 10 min. and will exit all three containers when integration test finished.

When you want to run the test in CI environment, and want to get exit code from the test, we can get exit code by adding an option like;

env DURATION=600 \
  docker compose -f compose.yml up \
    --abort-on-container-exit --exit-code-from client

When using --abort-on-container-exit option, server container will become a dirty status that abort server processes, which cause error when next run. It is recommended to use docker compose down to remove aborted container.

check logs with compose

You can check the execution log with docker-compose command after execution.

docker compose logs client

When you want to monitor log, you can add -f option for logs command.

docker compose logs -f client

Stop test setup

You can also stop test setup and remove image from another shell.

docker compose -f compose.yml stop

This stops container execution, and you can see logs afterword. This is as same as enter Ctrl-C key in executed shell.

Cleanup resources

After the end of tests, images of containers are on disk. You can clean up resources with a command

docker compose -f compose.yml down

This cleans up container resources.

repository type

When “TYPE” environment variable is set to “GIT”, it tests https and ssh+git protocols with a mapping configuration with http repository for source data.

When set it to “SVN”, it tests with https and ssh+svn protocols with a mapping configuration with http repository for source data.

When “TYPE” is not set or other than above, it tests with “GIT” configuration as default.

env DURATION=2400 TYPE=GIT docker compose up

env DURATION=2400 TYPE=SVN docker compose up

How to run with nerdctl

nerdctl supports similar syntax with docker, you can easily operate it.

For example,

env DURATION=2400 TYPE=SVN nerdctl compose up
nerdctl compose down

How to run with podman

Podman V3.0 support docker-compose through REST API. Please check Redhat manuals to enable REST API. See Using Podman and Docker Compose

runner shell script

For your convenience, we have a runner script test-integration/runner.sh. The script automatically detect docker, nerdctl or docker-compose for run.

you can run it like

test-integration/runner.sh GIT 1200

Manual execution

The way you do the integration test is:

  1. Create a team project.

  2. This can be a remote project (GitHub, etc.) or it can be local (file://…).

  3. For the latter case, you can use the OmegaT CLI team tool to easily init the project.

How to run

Prepare repository server

You need to prepare repository server where test against. It can not be empty, but an omegat team project will be overwritten by integration test.

Simple integration test

To run a simple integration test to check update integrity in concurrent edits, you can run it simply

./gradlew testIntegration -Domegat.test.repo=<URL of repo>.

  1. Specify the test duration with -Domegat.test.duration=X where X is a number of seconds (default is 14,400s = 4 hours)

  2. Wait

  3. Check the result

Mapping test

To run with mapping configuration, you can give mapping property to the test. An integration test prepares a team project with a specified mapping.

./gradlew testIntegration -Domegat.test.repo=<URL of repo> \
 -Domegat.test.map.repo=<URL of mapping target directory> \
 -Domegat.test.map.file=<Filename of mapping target>

For example, when you have a repository in GitHub user example repository omegat-test.git and mapping target https://example.com/index.html as source/index.html

./gradlew testIntegration -Domegat.test.repo=git@github.com:example/omegat-test.git \
 -Domegat.test.map.repo=https://example.com/ \
 -Domegat.test.map.file=index.html

Multiple scheme test

Some popular public git repository services such as GitHub and Gitlab support both ssh+git and git smart http(s) access for the repositories. You can test a situation that team project members access both protocols.

./gradlew testIntegration -Domegat.test.repo=<URL of repo> -Domegat.test.repo.alt=<another URL of repo>

It is worth testing with configuration of both mapping and multiple protocol scheme, Because OmegaT embeds a repository URL in remote omegat.project file when mapping feature is used, OmegaT v5.7 or before override a local access URL scheme by remote URL. When you clone team project with git+ssh to local work folder, then omegat access team repository, that is https URL in remote omegat.project, you are forced to use https instead of git+ssh.

An integration with a configuration of mapping and multiple protocol scheme will detect a behavior.

Team repository schema

OmegaT supports git and subversion repository URL.

Git URLs

Git Branches

Automatically detect default branch such as main or master It also accept arbitrary branch name. Integration test does not accept branch name.

Subversion URLs