Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

verify docker image #274

Merged
merged 2 commits into from
Sep 19, 2024
Merged

verify docker image #274

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f53ff32
verify docker image
salamonpavel Sep 18, 2024
b7eeee1
added comment about keyStorePath
salamonpavel Sep 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion project/Setup.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ object Setup {
}

val serverMergeStrategy = assembly / assemblyMergeStrategy := {
case PathList("META-INF", "services") => MergeStrategy.filterDistinctLines
case PathList("META-INF", "services", _*) => MergeStrategy.concat
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if there are same keys? (not sure what strategies there are)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the context of the sbt-assembly plugin, merge strategies determine how to handle files with the same path when creating an assembly JAR. Here are the common merge strategies:

MergeStrategy.concat

  • Purpose: Concatenates the contents of all files with the same path.
  • Usage: Useful when you want to combine the contents of multiple files into one, preserving all lines from each file.
  • Example: 
    case PathList("META-INF", "services", _*) => MergeStrategy.concat

MergeStrategy.singleOrError

  • Purpose: Ensures that only one file is included in the final assembly. If multiple files with the same path are found, it throws an error.
  • Usage: Useful when you need to guarantee that there is no ambiguity or conflict for a particular file.
  • Example: 
    case PathList("META-INF", "maven", "org.webjars", "swagger-ui", "pom.properties") => MergeStrategy.singleOrError

MergeStrategy.rename

  • Purpose: Handles file conflicts by renaming the conflicting files.
  • Usage: Useful when you want to include all versions of a file without merging their contents.
  • Example: 
    case PathList("META-INF", "services", _*) => MergeStrategy.rename

MergeStrategy.discard

  • Purpose: Discards all files with the same path.
  • Usage: Useful when you want to exclude certain files from the final assembly.
  • Example: 
    case PathList("META-INF", _*) => MergeStrategy.discard

MergeStrategy.first

  • Purpose: Keeps the first file encountered and discards the rest.
  • Usage: Useful when the order of files is known and you only need the first one.
  • Example: 
    case _ => MergeStrategy.first

MergeStrategy.last

  • Purpose: Keeps the last file encountered and discards the rest.
  • Usage: Useful when the order of files is known and you only need the last one.
  • Example: 
    case _ => MergeStrategy.last

MergeStrategy.filterDistinctLines

  • Purpose: Merges files by including only distinct lines.
  • Usage: Useful when you want to avoid duplicate lines in the merged file.
  • Example: 
    case "application.conf" => MergeStrategy.filterDistinctLines

MergeStrategy.deduplicate

  • Purpose: Merges files by including only distinct files.
  • Usage: Useful when you want to avoid duplicate files in the merged file.
  • Example: 
    case "reference.conf" => MergeStrategy.deduplicate

Custom Merge Strategy

  • Purpose: Allows you to define custom logic for merging files.
  • Usage: Useful when none of the predefined strategies fit your needs.
  • Example: 
    case "custom.conf" => new MergeStrategy {
  override def name: String = "customMergeStrategy"
  override def apply(tmp: File, path: String, files: Seq[File]): Either[String, Seq[File]] = {
    // Custom merge logic here
    Right(files)
  }
}

These strategies help manage how files are handled during the assembly process, ensuring that the final JAR is built according to your project's requirements.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have tested that it works also with filterDistinctLines. So let me know which one you prefer. @lsulak @benedeki

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MergeStrategy.filterDistinctLines and MergeStrategy.concat serve different purposes, and the choice between them depends on your specific requirements.

MergeStrategy.filterDistinctLines

  • Purpose: Merges files by including only distinct lines.
  • Usage: Useful when you want to avoid duplicate lines in the merged file.
  • Safety: Safer in scenarios where duplicate lines could cause issues, as it ensures each line appears only once.

MergeStrategy.concat

  • Purpose: Concatenates the contents of all files with the same path.
  • Usage: Useful when you want to combine the contents of multiple files into one, preserving all lines from each file.
  • Safety: May lead to conflicts or unexpected behavior if there are lines with the same keys but different values.

Conclusion

  • filterDistinctLines is generally safer when you need to ensure that there are no duplicate lines in the merged file.
  • concat is useful when you need to preserve all lines, but it requires careful handling of potential conflicts.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much for the educative window!

I can't tell you my preference because I don't know what duplicates we have here, and what lines. So I will base my answer according to these details - were you able to find out the specifics?

Also, was there a problem so that you changed it, or you just experimented with it? At first, I thought that filterDistinctLines was not working for us, that's why you changed it?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change wasn't just about the strategy. Honestly I didn't dig very deeply into this, just made it work by trial/error. At this moment I would rather dedicate my time to something more pressing than exploring this further.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understood, well then I don't really have preferences. Whatever works would be my answer, and if we encounter a problem around these, we can revisit it later with this knowledge ^ in mind 🤷

case PathList("META-INF", "maven", "org.webjars", "swagger-ui", "pom.properties") => MergeStrategy.singleOrError
case PathList("META-INF", "resources", "webjars", "swagger-ui", _*) => MergeStrategy.singleOrError
case PathList("META-INF", _*) => MergeStrategy.discard
Expand Down
22 changes: 22 additions & 0 deletions server/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,3 +48,25 @@ When monitoring enabled, the application exposes `http://localhost:8080/metrics`
and/or `http://localhost:8080/zio-metrics` endpoints which can be scraped by Prometheus.
For testing purposes there is [docker-compose.yml](./docker-compose.yml) file which can be used to start up dockerized
Prometheus and Grafana instances. Prometheus scraping configs are defined in [prometheus.yml](./prometheus.yml) file.

# Build & run docker image

```shell
sbt clean assembly

docker build -t absaoss/atum-service:latest --progress=plain --no-cache \
--build-arg BUILD_PROXY=http://zproxycloud.intra.absaafrica:80 \
--build-arg CONFIG=./src/main/resources/reference.conf \
--build-arg SSL=true \
--build-arg LDAP_SSL_CERTS_PATH=./certs \
--build-arg SSL_DNAME="CN=*.my.domain.com, OU=project1, O=mycorp, L=Johannesburg, ST=Gauteng, C=za" ./

docker run -p 8080:8080 -p 8443:8443 absaoss/atum-service:latest

# reference.conf file has to be configured as below when running with ssl enabled
ssl {
enabled=true
keyStorePassword=changeit
keyStorePath="/etc/ssl/certs/selfsigned.jks"
}
```
4 changes: 2 additions & 2 deletions server/src/main/resources/logback.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@
<Pattern>%d{HH:mm:ss.SSS} %-5level [%thread] %logger{36}:%L - %msg %ex{short}%n</Pattern>
</layout>
</appender>
<root level="debug">
<root level="info">
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

<appender-ref ref="STDOUT"/>
</root>
<logger name="za.co.absa" level="debug"/>
<logger name="za.co.absa" level="info"/>
<logger name="com.zaxxer.hikari" level="warn"/>
</configuration>
10 changes: 7 additions & 3 deletions server/src/main/resources/reference.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@
postgres {
# The JDBC driver class
dataSourceClass=org.postgresql.Driver
serverName=localhost // host.docker.internal for local run in docker against db on its host machine; localhost otherwise for testing and for the gh pipeline
# host.docker.internal for local run against db running in docker on its host machine; localhost otherwise for testing and for the gh pipeline
serverName=localhost
portNumber=5432
databaseName=atum_db
user=atum_user
Expand All @@ -16,8 +17,11 @@
}
ssl {
enabled=false
keyStorePassword=password
keyStorePath="/path/to/your/cert"
keyStorePassword=changeit
benedeki marked this conversation as resolved.
Show resolved Hide resolved
# location of Java Keystore (JKS) file with certificate to be used for HTTPS communication;
# it can be sufficient to use self-signed certificate on testing or development environment or
# when application runs in an internal network; otherwise certificates are usually obtained from a trusted CA
keyStorePath="/etc/ssl/certs/selfsigned.jks"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps a small comment for this value can be good to have here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lsulak do you have any suggestion for a good comment here?

Copy link
Collaborator

@lsulak lsulak Sep 19, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure, but perhaps something like this?

location of Java Keystore (JKS) file with certificate to be used for HTTPS communication; it can be sufficient to use self-signed certificate on testing or development environment or when application runs in an internal network; otherwise certificates are usually obtained from a trusted CA

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's a bit wordy but thanks will use it

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah I know, feel free to change it :D

}
monitoring {
# monitoring of http communication
Expand Down
Loading