Commit 3634a745 authored by Jean-Baptiste Nizet's avatar Jean-Baptiste Nizet
Browse files

feat: add a build property to build different apps

parent 74db26a2
......@@ -104,16 +104,16 @@ The application uses two physical indices:
The application doesn't use the physical resources index directly. Instead, it uses two aliases, that must be created
before using the application:
* `resource-index` is the alias used by the application to search for genetic resources
* `resource-harvest-index` is the alias used by the application to store genetic resources when the harvest is triggered.
* `rare-resource-index` is the alias used by the application to search for genetic resources
* `rare-resource-harvest-index` is the alias used by the application to store genetic resources when the harvest is triggered.
In normal operations, these two aliases should refer to the same physical resource index. The script
`createIndexAndAliases.sh` creates a physical index (named `resource-physical-index`) and creates these two aliases
`createIndexAndAliases.sh` creates a physical index (named `rare-resource-physical-index`) and creates these two aliases
referring to this physical index.
Once the index and the aliases have been created, a harvest can be triggered. The first operation that a harvest
does is to create or update (put) the mapping for the genetic resource entity into the index aliased by `resource-harvest-index`.
Then it parses the JSON files and stores them into this same index. Since the `resource-index` alias
does is to create or update (put) the mapping for the genetic resource entity into the index aliased by `rare-resource-harvest-index`.
Then it parses the JSON files and stores them into this same index. Since the `rare-resource-index` alias
normally refers to the same physical index, searches will find the resources stored by the harvester.
### Why two aliases
......@@ -144,14 +144,14 @@ very long time.
Here are curl commands illustrating the above scenario:
```
# delete the physical index and its aliases
curl -X DELETE "localhost:9200/resource-physical-index"
curl -X DELETE "localhost:9200/rare-resource-physical-index"
# recreate the physical index and its aliases
curl -X PUT "localhost:9200/resource-physical-index" -H 'Content-Type: application/json' -d'
curl -X PUT "localhost:9200/rare-resource-physical-index" -H 'Content-Type: application/json' -d'
{
"aliases" : {
"resource-index" : {},
"resource-harvest-index" : {}
"rare-resource-index" : {},
"rare-resource-harvest-index" : {}
}
}
'
......@@ -161,28 +161,28 @@ curl -X PUT "localhost:9200/resource-physical-index" -H 'Content-Type: applicati
If you don't want any downtime, you can instead use the following procedure:
- create a new physical index (let's name it `resource-new-physical-index`);
- delete the `resource-harvest-index` alias, and recreate it so that it refers to `resource-new-physical-index`;
- trigger a harvest. During the harvest, the `resource-index` alias, used by the search,
- create a new physical index (let's name it `rare-resource-new-physical-index`);
- delete the `rare-resource-harvest-index` alias, and recreate it so that it refers to `rare-resource-new-physical-index`;
- trigger a harvest. During the harvest, the `rare-resource-index` alias, used by the search,
still refers to the old physical index, and it thus still works flawlessly;
- once the harvest is finished, delete the `resource-index` alias, and recreate it so that it refers to
`resource-new-physical-index`. All the search operations will now use the new index, containing up-to-date
- once the harvest is finished, delete the `rare-resource-index` alias, and recreate it so that it refers to
`rare-resource-new-physical-index`. All the search operations will now use the new index, containing up-to-date
documents;
- delete the old physical index.
Here are curl commands illustrating the above scenario:
```
# create a new physical index
curl -X PUT "localhost:9200/resource-new-physical-index" -H 'Content-Type: application/json' -d'
curl -X PUT "localhost:9200/rare-resource-new-physical-index" -H 'Content-Type: application/json' -d'
{}
'
# delete the `resource-harvest-index` alias, and recreate it so that it refers to `resource-new-physical-index`
# delete the `rare-resource-harvest-index` alias, and recreate it so that it refers to `rare-resource-new-physical-index`
curl -X POST "localhost:9200/_aliases" -H 'Content-Type: application/json' -d'
{
"actions" : [
{ "remove" : { "index" : "resource-physical-index", "alias" : "resource-harvest-index" } },
{ "add" : { "index" : "resource-new-physical-index", "alias" : "resource-harvest-index" } }
{ "remove" : { "index" : "rare-resource-physical-index", "alias" : "rare-resource-harvest-index" } },
{ "add" : { "index" : "rare-resource-new-physical-index", "alias" : "rare-resource-harvest-index" } }
]
}
'
......@@ -191,14 +191,14 @@ curl -X POST "localhost:9200/_aliases" -H 'Content-Type: application/json' -d'
curl -X POST "localhost:9200/_aliases" -H 'Content-Type: application/json' -d'
{
"actions" : [
{ "remove" : { "index" : "resource-physical-index", "alias" : "resource-index" } },
{ "add" : { "index" : "resource-new-physical-index", "alias" : "resource-index" } }
{ "remove" : { "index" : "rare-resource-physical-index", "alias" : "rare-resource-index" } },
{ "add" : { "index" : "rare-resource-new-physical-index", "alias" : "rare-resource-index" } }
]
}
'
# delete the old physical index
curl -X DELETE "localhost:9200/resource-physical-index"
curl -X DELETE "localhost:9200/rare-resource-physical-index"
```
### Mapping migration
......@@ -210,9 +210,9 @@ of the application must be redeployed.
This is the easiest and safest procedure, that I would recommend:
- create a new physical index (let's name it `resource-new-physical-index`);
- delete the `resource-harvest-index` and the `resource-index` aliases, and recreate them both so that they refer to
`resource-new-physical-index`;
- create a new physical index (let's name it `rare-resource-new-physical-index`);
- delete the `rare-resource-harvest-index` and the `rare-resource-index` aliases, and recreate them both so that they refer to
`rare-resource-new-physical-index`;
- stop the existing application, deploy and start the new one;
- trigger a harvest;
- once everything is running fine, remove the old physical index.
......@@ -223,35 +223,35 @@ application can be restarted.
Here are curl commands illustrating the above scenario:
```
# create a new physical index
curl -X PUT "localhost:9200/resource-new-physical-index" -H 'Content-Type: application/json' -d'
curl -X PUT "localhost:9200/rare-resource-new-physical-index" -H 'Content-Type: application/json' -d'
{}
'
# delete the `resource-harvest-index` and the `resource-index` aliases, and recreate them both so that they refer to `resource-new-physical-index`
# delete the `rare-resource-harvest-index` and the `rare-resource-index` aliases, and recreate them both so that they refer to `rare-resource-new-physical-index`
curl -X POST "localhost:9200/_aliases" -H 'Content-Type: application/json' -d'
{
"actions" : [
{ "remove" : { "index" : "resource-physical-index", "alias" : "resource-harvest-index" } },
{ "add" : { "index" : "resource-new-physical-index", "alias" : "resource-harvest-index" } },
{ "remove" : { "index" : "resource-physical-index", "alias" : "resource-index" } },
{ "add" : { "index" : "resource-new-physical-index", "alias" : "resource-index" } }
{ "remove" : { "index" : "rare-resource-physical-index", "alias" : "resource-harvest-index" } },
{ "add" : { "index" : "rare-resource-new-physical-index", "alias" : "resource-harvest-index" } },
{ "remove" : { "index" : "rare-resource-physical-index", "alias" : "rare-resource-index" } },
{ "add" : { "index" : "rare-resource-new-physical-index", "alias" : "rare-resource-index" } }
]
}
'
# once everything is running fine, remove the old physical index.
curl -X DELETE "localhost:9200/resource-physical-index"
curl -X DELETE "localhost:9200/rare-resource-physical-index"
```
#### Upgrading with a very short downtime (or no downtime at all)
- create a new physical index (let's name it `resource-new-physical-index`);
- delete the `resource-harvest-index` alias, and recreate it so that it refers to `resource-new-physical-index`;
- delete the `rare-resource-harvest-index` alias, and recreate it so that it refers to `rare-resource-new-physical-index`;
- start the new application, on another machine, or on a different port, so that the new application code can be
used to trigger a harvest with the new schema, while the old application is still running and exposed to the users
- trigger the harvest on the **new** application
- once the harvest is finished, delete the `resource-index` alias, and recreate it so that it refers to
`resource-new-physical-index`;
- once the harvest is finished, delete the `rare-resource-index` alias, and recreate it so that it refers to
`rare-resource-new-physical-index`;
- expose the new application to the users instead of the old one
- stop the old application
......@@ -263,16 +263,16 @@ configuration to route to the new application once the harvest is done, for exam
Here are curl commands illustrating the above scenario:
```
# create a new physical index
curl -X PUT "localhost:9200/resource-new-physical-index" -H 'Content-Type: application/json' -d'
curl -X PUT "localhost:9200/rare-resource-new-physical-index" -H 'Content-Type: application/json' -d'
{}
'
# delete the `resource-harvest-index` alias, and recreate it so that it refers to `resource-new-physical-index`
# delete the `rare-resource-harvest-index` alias, and recreate it so that it refers to `rare-resource-new-physical-index`
curl -X POST "localhost:9200/_aliases" -H 'Content-Type: application/json' -d'
{
"actions" : [
{ "remove" : { "index" : "resource-physical-index", "alias" : "resource-harvest-index" } },
{ "add" : { "index" : "resource-new-physical-index", "alias" : "resource-harvest-index" } }
{ "remove" : { "index" : "rare-resource-physical-index", "alias" : "rare-resource-harvest-index" } },
{ "add" : { "index" : "rare-resource-new-physical-index", "alias" : "rare-resource-harvest-index" } }
]
}
'
......@@ -281,8 +281,8 @@ curl -X POST "localhost:9200/_aliases" -H 'Content-Type: application/json' -d'
curl -X POST "localhost:9200/_aliases" -H 'Content-Type: application/json' -d'
{
"actions" : [
{ "remove" : { "index" : "resource-physical-index", "alias" : "resource-index" } },
{ "add" : { "index" : "resource-new-physical-index", "alias" : "resource-index" } }
{ "remove" : { "index" : "rare-resource-physical-index", "alias" : "rare-resource-index" } },
{ "add" : { "index" : "rare-resource-new-physical-index", "alias" : "rare-resource-index" } }
]
}
'
......@@ -309,3 +309,32 @@ meaning the application has to be reboot if the configuration is changed on the
For a dynamic reload without restarting the application,
see http://cloud.spring.io/spring-cloud-static/Finchley.SR1/single/spring-cloud.html#refresh-scope
to check what has to be changed.
## Building other apps
By default, the built application is RARe. But this project actually allows building other
applications (WheatIS, for the moment, but more could come).
To build a different app, specify an `app` property when building. For example, to assemble
the WheatIS app, run the following command
./gradlew assemble -Papp=wheatis
You can also run the backend WheatIS application using
./gradlew bootRun -Papp=wheatis
Adding this property has the following consequences:
- the generated jar file (in `backend/build/libs`) is named `wheatis.jar` instead of `rare.jar`;
- the Spring active profile in `bootstrap.yml` is `wheatis-app` instead of `rare-app`;
- the frontend application built and embedded inside the jar file is the WheatIS frontend application instead of the RARe frontend application, i.e. the frontend command `yarn build:wheatis` is executed instead of the command `yarn:rare`.
Since the active Spring profile is different, all the properties specific to this profile
are applies. In particular:
- the context path of the application is `/wheatis` instead of `/rare`;
- the resource directory where the JSON files to harvest are looked up is different;
- the Elasticsearch prefix used for the index aliases is different.
See the `application.yml` file for details.
......@@ -40,6 +40,22 @@ tasks {
options.compilerArgs.add("-parameters")
}
getByName<Copy>("processResources") {
val app: String by project.extra
inputs.property("app", app)
filesMatching("bootstrap.yml") {
filter {
if (it.trim().startsWith("active:")) {
it.replace("rare", app)
}
else {
it
}
}
}
}
// this task is always out-of-date because it generates a properties file with the build time inside
// so the bootJar task is also always out of date, too, since it depends on it
// but it's better to do that than using the bootInfo() method of the springBoot closure, because that
......@@ -50,7 +66,8 @@ tasks {
}
val bootJar by getting(BootJar::class) {
archiveName = "rare.jar"
val app: String by project.extra
archiveName = "$app.jar"
dependsOn(":frontend:assemble")
dependsOn(buildInfo)
......
......@@ -3,4 +3,8 @@ spring:
name: rare
cloud:
config:
uri: ${SPRING_CONFIG_URI:http://localhost:8888}
\ No newline at end of file
uri: ${SPRING_CONFIG_URI:http://localhost:8888}
profiles:
# This default profile is replaced at gradle build time by the app passed using -Papp=wheatis for example
# Leave this line exactly as it is
active: rare-app
import org.gradle.tooling.BuildException
subprojects {
val app by this.extra {
val appProperty = project.findProperty("app") ?: "rare"
val acceptableValues = setOf("rare", "wheatis")
if (!acceptableValues.contains(appProperty)) {
throw BuildException("Invalid value for property app. Valid values are ${acceptableValues}.", null)
}
appProperty as String
}
}
import com.moowork.gradle.node.yarn.YarnTask
plugins {
base
id("com.moowork.node") version "1.2.0"
......@@ -28,7 +30,11 @@ tasks {
outputs.dir("node_modules")
}
val yarn_build by getting {
val yarn_build by creating(YarnTask::class) {
val app: String by project.extra
inputs.property("app", app)
args = listOf("build:${app}")
dependsOn(yarn_install)
inputs.dir("src")
outputs.dir("dist")
......@@ -52,7 +58,7 @@ tasks {
}
val clean by getting {
dependsOn("cleanYarn_run_build")
dependsOn("cleanYarn_build")
dependsOn("cleanYarn_run_test")
}
}
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment