Support #3608

Replacing text in file

Added by Vadim about 1 month ago. Updated 16 days ago.

Status:ClosedStart date:08/04/2020
Priority:NormalDue date:
Assignee:Vadim% Done:

0%

Category:-
Target version:RapidDeploy - 5.0-FIX
Affects Version:5.0-FIX Additional version details:
Timesheet Code:

Description

Hello Rafael.

Have faced new problem - cannot update data in a text file. "File Search/Replace" tasks didn't help me. Maybe I did something wrong...

My file content is such (part of a nginx proxy configuration):

location /__serviceShortName__ {
proxy_pass http://__serviceLongName__:8000/__serviceLongName__;
}
location /__serviceLongName__ {
proxy_pass http://__serviceLongName__:8000;

I need to replace both serviceShortName and serviceLongName with values that are loaded to the project on another steps. File will later be uploaded to a container.
Is there any task to handle such case?

Best regards,
Vadim.

search_replace_task_log (420 KB) Vadim, 08/10/2020 09:00 am

update_success (434 KB) Vadim, 08/10/2020 11:18 am

nginx_proxy_routing_configuration_update.zip (11.9 KB) Vadim, 08/10/2020 11:18 am

History

#1 Updated by Richard about 1 month ago

  • Assignee set to Rafael
  • Target version set to 5.0-FIX

#2 Updated by Rafael about 1 month ago

  • Status changed from New to Feedback
  • Assignee changed from Rafael to Vadim

Hi Vadim,

The "Search/Replace" tasks should do exactly what you need, what is the problem you are facing?

Let me know the configuration you are using to see if I can identify the issue.

Cheers!

Rafa

#3 Updated by Vadim about 1 month ago

Hi Rafael.

Hm... Something strange is happening today with with my work in RapidDeploy system. Just now I enabled this "Search/Replace" task and it passed without any problem. But before it just didn't do anything - no errors but also no changes in file. Now it's Ok. Sorry for disturbance. :) You can close ticket.

Best regards,
Vadim.

#4 Updated by Rafael about 1 month ago

  • Status changed from Feedback to Closed

Hi Vadim,

No problem! I wish all tickets were as easy as this one! :-P

Cheers!

Rafa

#5 Updated by Vadim about 1 month ago

Hi Rafael.

I'd like to ask You to reopen this issue. :)
Seems that something is wrong with something. Maybe my configuration is wrong, but it doesn't work again. All is left as it was. Last week I had some troubles with it - task was replacing "__serviceLongName__" key only. Now I get file without any changes at all. :) I don't know what to check. Can You please help me?

Bets regards,
Vadim.

#6 Updated by Rafael about 1 month ago

  • Assignee changed from Vadim to Rafael

#7 Updated by Richard about 1 month ago

  • Status changed from Closed to In Progress

#8 Updated by Rafael about 1 month ago

Hi Vadim,

Can you send me the log file from the successful deployment you previously had, and also if it's possible an export of the project?

Thanks!

Rafa

#9 Updated by Vadim about 1 month ago

Hi Rafael.

Here You are. Please find attached.

Best regards,
Vadim.

#10 Updated by Rafael about 1 month ago

Hi Vadim,

Here are my findings. I've created a sample scenario with a file called "location.conf" and the following content:

# Typical configuration file with a few parameters

__serviceLongName__=
__serviceShortName__=

When I run a deployment with a "Search/Replace Regex or Literal by File Type" task, I get the following output in the logs:


2020-08-12 12:54:00,827 [main] DEBUG com.midvision.rapiddeploy.orchestration.tasks.SearchReplaceTask - Search files with extensions: [conf]
2020-08-12 12:54:00,827 [main] DEBUG com.midvision.rapiddeploy.orchestration.tasks.SearchReplaceTask - Search archives with supported extensions: [tar, jar, zip, war, ear, rar, gzip]
2020-08-12 12:54:00,835 [main] DEBUG com.midvision.rapiddeploy.domain.deploy.pojo.EnvironmentPropertyFileServices - Key: __serviceShortName__ Value dms
2020-08-12 12:54:00,835 [main] DEBUG com.midvision.rapiddeploy.domain.deploy.pojo.EnvironmentPropertyFileServices - Key: __serviceLongName__ Value so_verification_service
2020-08-12 12:54:00,835 [main] DEBUG com.midvision.rapiddeploy.orchestration.tasks.SearchReplaceTask - Searching for: >__serviceShortName__< Replacing with: >dms<
2020-08-12 12:54:00,836 [main] INFO  com.midvision.rapiddeploy.utilities.io.sed -     File: /tmp/flt_nginx_templates/location.conf Replacing: >>__serviceShortName__\ *=.*<< with >>__serviceShortName__=dms<<
2020-08-12 12:54:00,836 [main] DEBUG com.midvision.rapiddeploy.orchestration.tasks.SearchReplaceTask - Searching for: >__serviceLongName__< Replacing with: >so_verification_service<
2020-08-12 12:54:00,838 [main] INFO  com.midvision.rapiddeploy.utilities.io.sed -     File: /tmp/flt_nginx_templates/location.conf Replacing: >>__serviceLongName__\ *=.*<< with >>__serviceLongName__=so_verification_service<<
2020-08-12 12:54:00,839 [main] INFO  com.midvision.rapiddeploy.orchestration.tasks.SearchReplaceTask - Task [ Search/Replace Regex or Literal by File Type ]...Complete...

As you can see, there are a couple of extra lines you are not getting:

INFO  com.midvision.rapiddeploy.utilities.io.sed -     File: /tmp/flt_nginx_templates/location.conf Replacing: >>__serviceShortName__\ *=.*<< with >>__serviceShortName__=dms<<
INFO  com.midvision.rapiddeploy.utilities.io.sed -     File: /tmp/flt_nginx_templates/location.conf Replacing: >>__serviceLongName__\ *=.*<< with >>__serviceLongName__=so_verification_service<<

If I deploy the project right after the first time and the values are already set, then I don't get any replacement, because the values are already set:

# Typical configuration file with a few parameters

__serviceLongName__=so_verification_service
__serviceShortName__=dms

If I change the values again:

# Typical configuration file with a few parameters

__serviceLongName__=OTHER_VALUE
__serviceShortName__=A_DIFFEREINT_VALUE

Then once again, after deploying, I get the replacements in the logs:

2020-08-12 13:18:19,412 [main] DEBUG com.midvision.rapiddeploy.domain.deploy.pojo.EnvironmentPropertyFileServices - Key: __serviceShortName__ Value dms
2020-08-12 13:18:19,412 [main] DEBUG com.midvision.rapiddeploy.domain.deploy.pojo.EnvironmentPropertyFileServices - Key: __serviceLongName__ Value so_verification_service
2020-08-12 13:18:19,412 [main] DEBUG com.midvision.rapiddeploy.orchestration.tasks.SearchReplaceTask - Searching for: >__serviceShortName__< Replacing with: >dms<
2020-08-12 13:18:19,413 [main] INFO  com.midvision.rapiddeploy.utilities.io.sed -     File: /tmp/flt_nginx_templates/location.conf Replacing: >>__serviceShortName__\ *=.*<< with >>__serviceShortName__=dms<<
2020-08-12 13:18:19,414 [main] DEBUG com.midvision.rapiddeploy.orchestration.tasks.SearchReplaceTask - Searching for: >__serviceLongName__< Replacing with: >so_verification_service<
2020-08-12 13:18:19,416 [main] INFO  com.midvision.rapiddeploy.utilities.io.sed -     File: /tmp/flt_nginx_templates/location.conf Replacing: >>__serviceLongName__\ *=.*<< with >>__serviceLongName__=so_verification_service<<
2020-08-12 13:18:19,416 [main] INFO  com.midvision.rapiddeploy.orchestration.tasks.SearchReplaceTask - Task [ Search/Replace Regex or Literal by File Type ]...Complete...

Conclusions

The reason you don't see any replacement could be because you either have the values already set in the "location.conf" file, or you don't have the "__serviceShortName__" and "__serviceLongName__" keys in that file. If you can send it to me, I can probably check what could be the reason.

I also noticed for the successful deployment you are using the following values:

Searching for: >__serviceShortName__< Replacing with: >dms<
Searching for: >__serviceLongName__< Replacing with: >document_management_service<

While for the failing deployment you are using different values:

Searching for: >__serviceShortName__< Replacing with: >svs<
Searching for: >__serviceLongName__< Replacing with: >so_verification_service<

I have no idea what these are for, but they could be the reason the Nginx test is failing.

I'm pretty sure the problem is not the replacemante but the test itself.

An easy debugging would be to copy the "location.conf" file into a string:

http://docs.midvision.com/LATEST/rapiddeploy-core-plugins-prd-generic/resources/filesystem/tasksMetaData.html#d2e3027

And then print it in the logs with an "Echo Message" task:

http://docs.midvision.com/LATEST/rapiddeploy-orchestration/resources/tasksMetaData.html#d1e1474

Before you actually copy it into the container, so we can properly know what's in the file.

I hope this helps, please let me know any update! :-)

Cheers!

Rafa

#11 Updated by Rafael about 1 month ago

  • Status changed from In Progress to Feedback
  • Assignee changed from Rafael to Vadim

Hi Vadim,

Please forget my entire previous update!

I just realised you had already pasted your "location.conf" file content and the problem is the "replaceAnyKeyValuewithValue" task resource!

Sorry for all the misleading... :-)

Imagine you have the following replacing "PROPERTY" entries:

__serviceLongName__  = so_verification_service
__serviceShortName__ = dms

If you leave the "replaceAnyKeyValuewithValue" resource set to "false", it will literally search for every appearance of "__serviceLongName__" and replace it for "so_verification_service" and every appearance of "__serviceShortName__" and replace it for "dms".

But if you set the "replaceAnyKeyValuewithValue" resource to "true", then it will look for properties inside the file and change the values for the keys matching the PROPERTY entry key.

Let's say, for example, you have a file with the following content:

__serviceLongName__=VALUE_A
__serviceShortName__=VALUE_B

And you have "replaceAnyKeyValuewithValue" set to "true", then the output would be:

__serviceLongName__=so_verification_service
__serviceShortName__=dms

But if you have "replaceAnyKeyValuewithValue" set to "false", then the output would be:

so_verification_service=VALUE_A
dms=VALUE_B

In your case, as you don't have properties inside the file to be replaced, there's no sense to have the "replaceAnyKeyValuewithValue" resource set to "true".

It was completely my fault, because I didn't see you had already pasted the content of the "location.conf" file, and because I saw the "replaceAnyKeyValuewithValue" resource set to "true" I completely assumed the file was a properties file, so sorry!! :-)

I hope everything is clearer now, please just forget my whole previous update in the ticket! :-)

Cheers!

Rafa

#12 Updated by Vadim about 1 month ago

Hi Rafael.

Thank You for the suggestions. Sorry, I didn't have enough time to test everything. I'll do it now.

Seems that I faced one more problem in this project. Something wrong is with Git repository copying. Yesterday and today this project deployments fail because final configuration testing fails. I tried to find out the problem and logs say that project operates right data and tries to update the configuration file. But... Configuration file is not updated from the repository. I even added "file delete" at the beginning of a project. Task deleted file, task with Git passes successfully, but there is no file in directory...
Git task is configured to clone repository, so it should update file from a remote source. Maybe there are some special settings?

Best regards,
Vadim.

#13 Updated by Rafael about 1 month ago

Hi Vadim,

Can you please create a new ticket for this new problem? :-)

This way we can have the different issues and solutions classified.

I'll investigate this though! ;-)

Cheers!

Rafa

#14 Updated by Vadim about 1 month ago

Hi Rafael.

Yes, of course the problem will be reported separately. just wanted to put it here as thought that those can be somehow linked.

Best regards,
Vadim.

#15 Updated by Rafael 26 days ago

  • Status changed from Feedback to Marked for Closure

Hi Vadim,

I saw you have already created a new ticket for the Git problem, please let me know if this ticket is still an issue, otherwise I'll just time it out in a few days time.

Cheers!

Rafa

#16 Updated by Rafael 16 days ago

  • Status changed from Marked for Closure to Closed

Also available in: Atom PDF