-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unable to cache https pages #11
Comments
I've never seen an issue like the one you report with HTTPS enabled. I have these jobs running many times daily on many sites, both with and without HTTPS. As for the first problem, I think it's rather simple. You have HTTPS disabled, but it appears links are being found in your site/theme/post/etc which are hard set to HTTPS. |
this is a really large site that I am migrating to azure. the All-In-One WP backup file was 2.5gig. I had to write a plugin to migrate the media file to blob storage. It could be the pages came in from the migration with hardcoded urls. I will check the tables to see if that is true. |
I would add, I have stopped using the cache loader on some larger sites which are operating on Premium app service plans with dedicated vCores for SQL DB. There weren't any issues, but we determined that a significant number of pages were being placed into cache which were never being accessed. With the performance of Premium and the SQL Elastic Pool, the cost to generate the page fresh and have it stored in cache at that time was minimal. |
For this site I ended up needing to do regular WP with a MySql backend. They had a very large number of plugins and I could not get ProjectNami working with it :( When I have the cache on I am definitely seeing an improvement to performance. |
Note that the cache does work without the loader. You just have the inital cost of generating expired pages, which then are written to the cache and the TTLs set. How often does page content need to be regenerated? You can set the TTLs for a few hours or even longer. Plus, if a page or post is edited in WP Admin, the plugin forces a cache purge for it immediately anyway. And as you have probably noticed, site admins can manually clear individual pages from cache by visiting them while logged in. The button will be in the Admin bar. |
Yes I did notice that nice feature. I think maybe this might be an issue in the way the guid id lookup is happening on the |
I was able to get it working by upgrading the project to .net 4.72 and update all the NuGet Packages. |
I can think of one reason that would have been the solution. By any chance is only TLS 1.2 enabled on your web app? .NET 4.7 is required for TLS 1.2 to be the default HTTPS method from apps. I will update the project here on GitHub accordingly. |
Yes it is. I think it is now the default when you add a new Web App. |
If I turn off https on my site I am able to cache pages but the webjob fails when site is set to https.
Webjob logs
With HTTPS turned off -still gtt some errors but it will cache like 12 pages
12/27/2019 22:16:37 > 8aad20: SYS INFO] Status changed to Initializing [12/27/2019 22:16:39 > 8aad20: SYS INFO] Run script 'ProjectNamiCacheLoader.exe' with script host - 'WindowsScriptHost' [12/27/2019 22:16:40 > 8aad20: SYS INFO] Status changed to Running [12/27/2019 22:16:45 > 8aad20: INFO] Visited - http://rentprep.azurewebsites.net [12/27/2019 22:16:51 > 8aad20: INFO] Visited - http://rentprep.azurewebsites.net/ [12/27/2019 22:16:54 > 8aad20: INFO] Visited - http://rentprep.azurewebsites.net/tenant-background-check-2/ [12/27/2019 22:16:56 > 8aad20: INFO] Visited - http://rentprep.azurewebsites.net/tenant-screening/credit-check/ [12/27/2019 22:16:58 > 8aad20: INFO] Visited - http://rentprep.azurewebsites.net/tenant-screening/smartmove/ [12/27/2019 22:17:00 > 8aad20: INFO] Visited - http://rentprep.azurewebsites.net/tenant-screening/high-volume/ [12/27/2019 22:17:00 > 8aad20: INFO] ERROR Get Response - The underlying connection was closed: An unexpected error occurred on a send. at System.Net.HttpWebRequest.GetResponse() [12/27/2019 22:17:00 > 8aad20: INFO] at ProjectNamiCacheLoader.Module1.GetResponse(String LinkURL) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 112 [12/27/2019 22:17:00 > 8aad20: INFO] ERROR Unable to visit https://rentprep.azurewebsites.net/about/screening-api/ - Object reference not set to an instance of an object. at ProjectNamiCacheLoader.Module1.VisitLink(String LinkURL, Int32 LinkDepth) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 37 [12/27/2019 22:17:02 > 8aad20: INFO] Visited - http://rentprep.azurewebsites.net/tenant-screening/ [12/27/2019 22:17:05 > 8aad20: INFO] Visited - http://rentprep.azurewebsites.net/tenant-screening/sample-report/ [12/27/2019 22:17:07 > 8aad20: INFO] Visited - http://rentprep.azurewebsites.net/tenant-screening/order/ [12/27/2019 22:17:09 > 8aad20: INFO] Visited - http://rentprep.azurewebsites.net/about/ [12/27/2019 22:17:11 > 8aad20: INFO] Visited - http://rentprep.azurewebsites.net/about/contact/ [12/27/2019 22:17:13 > 8aad20: INFO] Visited - http://rentprep.azurewebsites.net/resources/ [12/27/2019 22:17:16 > 8aad20: INFO] Visited - http://rentprep.azurewebsites.net/blog/ [12/27/2019 22:17:18 > 8aad20: INFO] Visited - http://rentprep.azurewebsites.net/podcast/ [12/27/2019 22:17:20 > 8aad20: INFO] Visited - http://rentprep.azurewebsites.net/group/ [12/27/2019 22:17:22 > 8aad20: INFO] Visited - http://rentprep.azurewebsites.net/landlord-tenant-laws/ [12/27/2019 22:17:24 > 8aad20: INFO] Visited - http://rentprep.azurewebsites.net/resources/renters-relations/ [12/27/2019 22:17:26 > 8aad20: INFO] Visited - http://rentprep.azurewebsites.net/resources/faq/ [12/27/2019 22:17:28 > 8aad20: INFO] Visited - http://rentprep.azurewebsites.net/tenant-screening/packages-pricing/ [12/27/2019 22:17:28 > 8aad20: INFO] ERROR Get Response - The underlying connection was closed: An unexpected error occurred on a send. at System.Net.HttpWebRequest.GetResponse() [12/27/2019 22:17:28 > 8aad20: INFO] at ProjectNamiCacheLoader.Module1.GetResponse(String LinkURL) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 112 [12/27/2019 22:17:28 > 8aad20: INFO] ERROR Unable to visit https://rentprep.azurewebsites.net/tenant-screening/packages-pricing/ - Object reference not set to an instance of an object. at ProjectNamiCacheLoader.Module1.VisitLink(String LinkURL, Int32 LinkDepth) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 37 [12/27/2019 22:17:28 > 8aad20: INFO] ERROR Get Response - The underlying connection was closed: An unexpected error occurred on a send. at System.Net.HttpWebRequest.GetResponse() [12/27/2019 22:17:28 > 8aad20: INFO] at ProjectNamiCacheLoader.Module1.GetResponse(String LinkURL) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 112 [12/27/2019 22:17:28 > 8aad20: INFO] ERROR Unable to visit https://rentprep.azurewebsites.net/tenant-screening/ - Object reference not set to an instance of an object. at ProjectNamiCacheLoader.Module1.VisitLink(String LinkURL, Int32 LinkDepth) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 37 [12/27/2019 22:17:28 > 8aad20: INFO] ERROR Get Response - The underlying connection was closed: An unexpected error occurred on a send. at System.Net.HttpWebRequest.GetResponse() [12/27/2019 22:17:28 > 8aad20: INFO] at ProjectNamiCacheLoader.Module1.GetResponse(String LinkURL) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 112 [12/27/2019 22:17:28 > 8aad20: INFO] ERROR Unable to visit https://rentprep.azurewebsites.net/tenant-screening/rental-background-check/ - Object reference not set to an instance of an object. at ProjectNamiCacheLoader.Module1.VisitLink(String LinkURL, Int32 LinkDepth) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 37 [12/27/2019 22:17:28 > 8aad20: INFO] ERROR Get Response - The underlying connection was closed: An unexpected error occurred on a send. at System.Net.HttpWebRequest.GetResponse() [12/27/2019 22:17:28 > 8aad20: INFO] at ProjectNamiCacheLoader.Module1.GetResponse(String LinkURL) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 112 [12/27/2019 22:17:28 > 8aad20: INFO] ERROR Unable to visit https://rentprep.azurewebsites.net/tenant-screening/credit-check/ - Object reference not set to an instance of an object. at ProjectNamiCacheLoader.Module1.VisitLink(String LinkURL, Int32 LinkDepth) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 37 [12/27/2019 22:17:28 > 8aad20: INFO] ERROR Get Response - The underlying connection was closed: An unexpected error occurred on a send. at System.Net.HttpWebRequest.GetResponse() [12/27/2019 22:17:28 > 8aad20: INFO] at ProjectNamiCacheLoader.Module1.GetResponse(String LinkURL) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 112 [12/27/2019 22:17:28 > 8aad20: INFO] ERROR Unable to visit https://rentprep.azurewebsites.net/rentprep-reviews/ - Object reference not set to an instance of an object. at ProjectNamiCacheLoader.Module1.VisitLink(String LinkURL, Int32 LinkDepth) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 37 [12/27/2019 22:17:28 > 8aad20: INFO] ERROR Get Response - The underlying connection was closed: An unexpected error occurred on a send. at System.Net.HttpWebRequest.GetResponse() [12/27/2019 22:17:28 > 8aad20: INFO] at ProjectNamiCacheLoader.Module1.GetResponse(String LinkURL) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 112 [12/27/2019 22:17:28 > 8aad20: INFO] ERROR Unable to visit https://rentprep.azurewebsites.net/resources/landlord-forms/ - Object reference not set to an instance of an object. at ProjectNamiCacheLoader.Module1.VisitLink(String LinkURL, Int32 LinkDepth) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 37 [12/27/2019 22:17:28 > 8aad20: INFO] ERROR Get Response - The underlying connection was closed: An unexpected error occurred on a send. at System.Net.HttpWebRequest.GetResponse() [12/27/2019 22:17:28 > 8aad20: INFO] at ProjectNamiCacheLoader.Module1.GetResponse(String LinkURL) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 112 [12/27/2019 22:17:28 > 8aad20: INFO] ERROR Unable to visit https://rentprep.azurewebsites.net/blog/ - Object reference not set to an instance of an object. at ProjectNamiCacheLoader.Module1.VisitLink(String LinkURL, Int32 LinkDepth) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 37 [12/27/2019 22:17:28 > 8aad20: INFO] ERROR Get Response - The underlying connection was closed: An unexpected error occurred on a send. at System.Net.HttpWebRequest.GetResponse() [12/27/2019 22:17:28 > 8aad20: INFO] at ProjectNamiCacheLoader.Module1.GetResponse(String LinkURL) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 112 [12/27/2019 22:17:28 > 8aad20: INFO] ERROR Unable to visit https://rentprep.azurewebsites.net/podcast/ - Object reference not set to an instance of an object. at ProjectNamiCacheLoader.Module1.VisitLink(String LinkURL, Int32 LinkDepth) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 37 [12/27/2019 22:17:28 > 8aad20: INFO] ERROR Get Response - The underlying connection was closed: An unexpected error occurred on a send. at System.Net.HttpWebRequest.GetResponse() [12/27/2019 22:17:28 > 8aad20: INFO] at ProjectNamiCacheLoader.Module1.GetResponse(String LinkURL) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 112 [12/27/2019 22:17:28 > 8aad20: INFO] ERROR Unable to visit https://rentprep.azurewebsites.net/resources/ - Object reference not set to an instance of an object. at ProjectNamiCacheLoader.Module1.VisitLink(String LinkURL, Int32 LinkDepth) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 37 [12/27/2019 22:17:28 > 8aad20: INFO] ERROR Get Response - The underlying connection was closed: An unexpected error occurred on a send. at System.Net.HttpWebRequest.GetResponse() [12/27/2019 22:17:28 > 8aad20: INFO] at ProjectNamiCacheLoader.Module1.GetResponse(String LinkURL) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 112 [12/27/2019 22:17:28 > 8aad20: INFO] ERROR Unable to visit https://rentprep.azurewebsites.net/about/contact/ - Object reference not set to an instance of an object. at ProjectNamiCacheLoader.Module1.VisitLink(String LinkURL, Int32 LinkDepth) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 37 [12/27/2019 22:17:28 > 8aad20: INFO] ERROR Get Response - The underlying connection was closed: An unexpected error occurred on a send. at System.Net.HttpWebRequest.GetResponse() [12/27/2019 22:17:28 > 8aad20: INFO] at ProjectNamiCacheLoader.Module1.GetResponse(String LinkURL) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 112 [12/27/2019 22:17:28 > 8aad20: INFO] ERROR Unable to visit https://rentprep.azurewebsites.net/news/community/ - Object reference not set to an instance of an object. at ProjectNamiCacheLoader.Module1.VisitLink(String LinkURL, Int32 LinkDepth) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 37 [12/27/2019 22:17:28 > 8aad20: INFO] ERROR Get Response - The underlying connection was closed: An unexpected error occurred on a send. at System.Net.HttpWebRequest.GetResponse() [12/27/2019 22:17:28 > 8aad20: INFO] at ProjectNamiCacheLoader.Module1.GetResponse(String LinkURL) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 112 [12/27/2019 22:17:28 > 8aad20: INFO] ERROR Unable to visit https://rentprep.azurewebsites.net/tenant-screening/affiliate-program/ - Object reference not set to an instance of an object. at ProjectNamiCacheLoader.Module1.VisitLink(String LinkURL, Int32 LinkDepth) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 37 [12/27/2019 22:17:30 > 8aad20: INFO] Visited - http://rentprep.azurewebsites.net/tenant-screening/testimonials/ [12/27/2019 22:17:32 > 8aad20: INFO] Visited - http://rentprep.azurewebsites.net/about/security-privacy/ [12/27/2019 22:17:32 > 8aad20: INFO] ERROR Get Response - The underlying connection was closed: An unexpected error occurred on a send. at System.Net.HttpWebRequest.GetResponse() [12/27/2019 22:17:32 > 8aad20: INFO] at ProjectNamiCacheLoader.Module1.GetResponse(String LinkURL) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 112 [12/27/2019 22:17:32 > 8aad20: INFO] ERROR Unable to visit https://rentprep.azurewebsites.net/tenant-screening/questions-to-ask-potential-tenants/ - Object reference not set to an instance of an object. at ProjectNamiCacheLoader.Module1.VisitLink(String LinkURL, Int32 LinkDepth) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 37 [12/27/2019 22:17:32 > 8aad20: INFO] ERROR Get Response - The underlying connection was closed: An unexpected error occurred on a send. at System.Net.HttpWebRequest.GetResponse() [12/27/2019 22:17:32 > 8aad20: INFO] at ProjectNamiCacheLoader.Module1.GetResponse(String LinkURL) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 112 [12/27/2019 22:17:32 > 8aad20: INFO] ERROR Unable to visit https://rentprep.azurewebsites.net/tenant-screening/tenant-screening-services/ - Object reference not set to an instance of an object. at ProjectNamiCacheLoader.Module1.VisitLink(String LinkURL, Int32 LinkDepth) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 37 [12/27/2019 22:17:32 > 8aad20: INFO] ERROR Get Response - The underlying connection was closed: An unexpected error occurred on a send. at System.Net.HttpWebRequest.GetResponse() [12/27/2019 22:17:32 > 8aad20: INFO] at ProjectNamiCacheLoader.Module1.GetResponse(String LinkURL) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 112 [12/27/2019 22:17:32 > 8aad20: INFO] ERROR Unable to visit https://rentprep.azurewebsites.net/screening-services/tenant-background-check-services-reviews/ - Object reference not set to an instance of an object. at ProjectNamiCacheLoader.Module1.VisitLink(String LinkURL, Int32 LinkDepth) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 37 [12/27/2019 22:17:32 > 8aad20: SYS INFO] Status changed to Success
With HTTPS turned on - will not cache any pages
[12/27/2019 22:56:31 > 8aad20: SYS INFO] Status changed to Initializing [12/27/2019 22:56:33 > 8aad20: SYS INFO] Run script 'ProjectNamiCacheLoader.exe' with script host - 'WindowsScriptHost' [12/27/2019 22:56:33 > 8aad20: SYS INFO] Status changed to Running [12/27/2019 22:56:34 > 8aad20: INFO] ERROR Get Response - The underlying connection was closed: An unexpected error occurred on a send. at System.Net.HttpWebRequest.GetResponse() [12/27/2019 22:56:34 > 8aad20: INFO] at ProjectNamiCacheLoader.Module1.GetResponse(String LinkURL) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 112 [12/27/2019 22:56:34 > 8aad20: INFO] ERROR Unable to visit https://rentprep.azurewebsites.net - Object reference not set to an instance of an object. at ProjectNamiCacheLoader.Module1.VisitLink(String LinkURL, Int32 LinkDepth) in C:\VS2013_Projects\ProjectNamiCacheLoader\ProjectNamiCacheLoader\Module1.vb:line 37 [12/27/2019 22:56:34 > 8aad20: SYS INFO] Status changed to Success
The text was updated successfully, but these errors were encountered: