RK-J

findings(Current implementation):
sitemap filename - sitemap-subject-book-version.xml:
sitemap-python-3.9-library-reference-(dt)-prd.xml

• After UTF-8 encoding in backend:*
  sitemap-python-3.9-library-reference-%28dt%29-prd.xml

• sitemap filename for the book uploaded to edutestqa_sitemap bucket:*
  sitemap-python-3.9-library-reference-%28dt%29-prd.xml
  - In sitemap-python-3.9-library-reference-%28dt%29-prd.xml file,
  public URL - https://storage.googleapis.com/edutestqa_sitemap/sitemap-python-3.9-library-reference-%2528dt%2529-prd.xml
  - It is working and it is double encoded by the GCP.
  (iv)
  sitemap entry for the book present in sitemap.xml:
  - https://edutestqa.appspot.com/sitemap/edutestqa_sitemap/sitemap-python-3.9-library-reference-%28dt%29-prd.xml (not working)

Tried double encoding the sitemap filename for above URL:
- https://edutestqa.appspot.com/sitemap/edutestqa_sitemap/sitemap-python-3.9-library-reference-%2528dt%2529-prd.xml (not working)

Tried encoding the double encoded filename:(triple encoding)
- https://edutestqa.appspot.com/sitemap/edutestqa_sitemap/sitemap-python-3.9-library-reference-%252528dt%252529-prd.xml (working)

For URL that contains non-alphanumeric characters(which are double encoded)
- URL starts with https://storage.googleapis.com/edutestqa_sitemap/ is working
- URL starts with https://edutestqa.appspot.com/sitemap/edutestqa_sitemap/ is not working

conclusion with ex:
This URL(https://edutestqa.appspot.com/sitemap/edutestqa_sitemap/sitemap-python-3.9-library-reference-%28dt%29-prd.xml) is present in sitemap entry for the book in sitemap.xml. In the current implementation, it is doing UTF-8 encoding for filename. The sitemap URL is not working for the filename that has non-alphanumeric characters.
So, If we want the URL with non-alphanumeric characters to work, we have to encode the double encoded filename and use that URL for sitemap.xml.
i.e https://edutestqa.appspot.com/sitemap/edutestqa_sitemap/sitemap-python-3.9-library-reference-%252528dt%252529-prd.xml

Fix:

In the current implementation filename is stored in the bucket as:
sitemap-vb2583-tutorial-sitemap-%28test%29-2747.xml
So, we have to do the triple encoding to make the URL work i.e (sitemap-vb2583-tutorial-sitemap-%252528test%252529-2747.xml). This will fix the issue. But if the filename has many special characters it will increase the no of characters in URL. It is not efficient

So, To fix this:
we need to save the filename without any encoding to bucket :
sitemap-vb2583-tutorial-sitemap-(test)-2747.xml

and we do the UTF-8 encoding on filename only for the sitemapURL
https://edutestdev-240612.appspot.com/sitemap/edutestdev_sitemap/sitemap-vb2583-tutorial-sitemap-%28test%29-2747.xml

In this way, we can solve the issue.It is working.