I will try implementing your suggestions and see where that takes me.
which I am running now. I try to support as up-to-date protocols and
IE9+. The current ssllabs score is A.
when failing to achieve an acceptable score on JDK8. The only thing I
JDK11, which I did not succeed in yet. I put in the
not calling it in 9.4.14.
Thanks again for clearing most of that up. I will follow your
can do for me and try do see what I need to do about IE9+/TLS1.0.
Post by Joakim ErdfeltInline ...
On Tue, Nov 20, 2018 at 9:26 AM Silvio Bierman
Hello Joakim,
/***** start code *****/
val sslContextFactory = new SslContextFactory
      val ciphers = Array(
         "TLS_AES_128_GCM_SHA256", /* TLS 1.3 */
         "TLS_AES_256_GCM_SHA384", /* TLS 1.3 */
"TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384",
"TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
"TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384",
"TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
"TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
"TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
"TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA", /* TLS 1.0 / 1.1 - IE7/8/9 -
Android <= 4.3 */
"TLS_DHE_DSS_WITH_AES_256_GCM_SHA384",
"TLS_DHE_DSS_WITH_AES_128_GCM_SHA256",
"TLS_DHE_DSS_WITH_AES_256_CBC_SHA256")
sslContextFactory.setCipherComparator(HTTP2Cipher.COMPARATOR)
sslContextFactory.setUseCipherSuitesOrder(true)
Not relevant, anywhere anymore.
You can get an A+ rating on ssllabs without attempting to control
cipher suite order.
(Note: A+ rating is for all HTTP/1.x and HTTP/2 behaviors)
sslContextFactory.setIncludeCipherSuites(ciphers:_*)
Don't include, exclude only.
You eliminate a ton of valid Cipher Suites doing it this way.
sslContextFactory.setExcludeCipherSuites("SSL_RSA_WITH_DES_CBC_SHA",
"SSL_DHE_RSA_WITH_DES_CBC_SHA",
"SSL_DHE_DSS_WITH_DES_CBC_SHA",
"SSL_RSA_EXPORT_WITH_RC4_40_MD5",
"SSL_RSA_EXPORT_WITH_DES40_CBC_SHA",
"SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA",
"SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA")
Use defaults, none of those Cipher suites would be selected anyway due
to default Cipher Suite exclusions in SslContextFactory AND the JVM
disabled cipher suites.
https://github.com/eclipse/jetty.project/blob/jetty-9.4.14.v20181114/jetty-util/src/main/java/org/eclipse/jetty/util/ssl/SslContextFactory.java#L135-L148
 /** Default Excluded Cipher Suite List */
 private static final String[] DEFAULT_EXCLUDED_CIPHER_SUITES = {
     // Exclude weak / insecure ciphers
     "^.*_(MD5|SHA|SHA1)$",
     // Exclude ciphers that don't support forward secrecy
     "^TLS_RSA_.*$",
     // The following exclusions are present to cleanup known bad
cipher
     // suites that may be accidentally included via include
patterns.
     // The default enabled cipher list in Java will not include
these
     // (but they are available in the supported list).
     "^SSL_.*$",
     "^.*_NULL_.*$",
     "^.*_anon_.*$"
 };
sslContextFactory.addExcludeProtocols("SSLv3")
This is the in the default excluded protocols values, and also
excluded/disabled at the JVM level, don't set it.
https://github.com/eclipse/jetty.project/blob/jetty-9.4.14.v20181114/jetty-util/src/main/java/org/eclipse/jetty/util/ssl/SslContextFactory.java#L132-L133
 /** Default Excluded Protocols List */
 private static final String[] DEFAULT_EXCLUDED_PROTOCOLS = {"SSL",
"SSLv2", "SSLv2Hello", "SSLv3"};
sslContextFactory.setRenegotiationAllowed(false)
sslContextFactory.setEndpointIdentificationAlgorithm("HTTPS")
This is only relevant for a client initiating a connection to a
destination machine with TLS.
Comment out all of the above.
It screams old-school behaviors and techniques.
Use defaults for SslContextFactory when it comes to Cipher Suites and
Protocols, and don't attempt to support TLS/1.0 or TLS/1.1 Those
protocols are disabled in so many places (User-Agents, Browsers, OS
level, Java VM level, Server level, various hardware intermediates)
now its pointless to attempt to support them.  If this is still
important to you, make a different connector for legacy behaviors.
sslContextFactory.setKeyStorePath(keyStore)
sslContextFactory.setKeyStorePassword(keyPassword)
sslContextFactory.setKeyManagerPassword(keyPassword)
sslContextFactory.setKeyStoreType(keyStoreType)
      val httpsConfig = new HttpConfiguration
httpsConfig.setSecureScheme("https")
httpsConfig.setSecurePort(port)
httpsConfig.setSendXPoweredBy(false)
httpsConfig.setSendServerVersion(false)
httpsConfig.setOutputBufferSize(32768)
      httpsConfig.addCustomizer(new SecureRequestCustomizer)
      val http1 = new HttpConnectionFactory(httpsConfig)
      val connector =
      {
         if (httpVersion != "http/2")
         {
            val ssl = new
SslConnectionFactory(sslContextFactory,"http/1.1")
            val http = new HttpConnectionFactory(httpsConfig)
            new ServerConnector(server,ssl,http)
         }
         else
         {
            val http2 = new
HTTP2ServerConnectionFactory(httpsConfig)
         val alpn = new ALPNServerConnectionFactory
alpn.setDefaultProtocol(http1.getProtocol)
Use alpn.setDefaultProtocol(http1.getDefaultProtocol())
val ssl = new SslConnectionFactory(sslContextFactory,alpn.getProtocol)
            new ServerConnector(server,ssl,alpn,http2,http1)
         }
      }
      if (address != null) connector.setHost(address)
      connector.setPort(port)
      connector.setIdleTimeout(1000L * idleTimeout)
      server.addConnector(connector)
/***** end code *****/
/***** start code *****/
   val pool = new
QueuedThreadPool(maxThreads,minThreads,idleTimeout,new
ArrayBlockingQueue(queueSize))
   val server = new Server(pool)
/***** end code *****/
HTTP/2 will seriously stress out your thread pool.
Don't specify the queue, use defaults.
Don't use less then defaults for min/max threads with HTTP/2.
It's good that you are using the QueuedThreadPool, it behaves much
better then a raw Executor with regards to HTTP/2.
Threads are 10-100, idletimeouts I have tried are from 1000 to
20000, queueSize is now 500 but I have tried both much smaller and
larger values.
I have no idea what values would be sensible. The number of
concurrent users is relatively low: up to a couple of hundred,
rarely more. Users can do requests that take longish to complete
(sometimes up to 30 or even 60 seconds) but those are very rare.
The majority is relatively quick (< 50ms). Current load on the
low memory and CPU close to zero.
No problems with HTTP/1.1 with this code/settings, problematic
with HTTP/2. Strangely enough some servers seem fine (or users
just did not report any problems yet).
Start by using a default QueuedThreadPool, no configuration, no min
thread, no max threads, all strictly defaults.
Then monitor your usages and behaviors with HTTP/2 enabled over at
least a week.
Then you can tune your maxThreads.
As to thread idle timeouts, don't bother setting them yet.
Your connector.setIdleTimeout() is far more useful to you.
Any help would be welcome.
Kind regards,
Silvio
Many folks have gotten HTTP/2 working just fine in embedded-jetty.
1. Not having alpn support setup properly
2. Too restrictive of an SslContextFactory that is incompatible with HTTP/2
When it comes to the ALPN suppport, you have to worry about what JVM
runtime you are on first.
If you are using Java 8 then you have to align your alpn-boot version
to the specific Java 8 JVM you are using.
https://github.com/jetty-project/jetty-alpn/blob/master/docs/version_mapping.properties
Under Java 8, you MUST use the -Xbootclasspath on JVM startup to have
alpn work properly.
However, if you are using Java 11 then you can use the
jetty-alpn-java-server artifact instead (no bootclasspath requirement)
to get ALPN behaviors required for HTTP/2.
You also have a conscrypt option for running ALPNÂ + HTTP/2 on Jetty,
which will work the same regardless of Java 8 or Java 11.
When it comes to too restrictive of an SslContextFactory we have the
* If you are specifying Include Cipher Suites you are too restrictive.
* Your certificates need to be RSA (not DES) and at a higher bit size
(like 2048+)
* If you are attempting to get old Protocols (TLS/1.0 TLS/1.1)
working, then you are introducing incompatible Cipher Suites too.
* If you are resetting the Exclude Cipher Suites you are introducing
incompatible Cipher Suites.
* Avoid using Include Cipher Suites (this exists for people stuck in
the past and not ever wanting to use HTTP/2)
* Avoid using Include Protocols (this exists for people stuck
supporting old clients and not wanting to use HTTP/2)
* Add to the existing Exclude Cipher Suites (if needed)
* Add to the existing Exclude Protocols (if needed)
* Understand the OpenJDK JVM Crypto Roadmap and its implications -
https://java.com/en/jre-jdk-cryptoroadmap.html
* Always use an Up to date (not-expired) JVM
* Learn how to use the SslContextFactory.dump() methods at runtime to
know what the current state of your SslContextFactory is (including
all selected cipher suites and protocols)
* Test your running server on https://www.ssllabs.com/ssltest/ to see
where you are still having configuration issues with SslContextFactory
We have an example codebase that shows how to start an HTTP/2 server
connector.
https://github.com/eclipse/jetty.project/blob/jetty-9.4.x/examples/embedded/src/main/java/org/eclipse/jetty/embedded/Http2Server.java
You can check the Jenkins project for their usage of HTTP/2 on
Embedded Jetty as well. (their self executing WAR file does just this)
You could also run h2spec against your server and see if its behaving
properly.
https://github.com/summerwind/h2spec
- Joakim
Post by Joakim ErdfeltHow do you enable HTTP/2 in your embedded application?
Also, If you have configured your SslContextFactory, what does
this configuration look like?
Lastly, what does your ThreadPool / Executor configuration look like?
On Tue, Nov 20, 2018 at 9:01 AM Silvio Bierman
Hello all,
We are running Jetty 9.4.14 embedded in an application that runs on a
number of Ubuntu 18.10 servers. Last night we switched from HTTP1.1 only
to HTTP2. Today we are experiencing serious problems. Users can not
access the application or are experiencing extremely long latencies. At
the same time the server is almost idle.
We have configured the Ubuntu servers as almost default setups.
OpenJDK11 with Jetty using pure Java HTTPS. We have been doing this for
years now and each time we switched from HTTP1.1 to HTTP2 things started
falling apart. In the past we had reports of people not being able to
connect at all receiving spurious messages from their
browsers (some of
which are undoubtedly outdated). We expected things to have improved by
now and decided to try again, with the above result.
Are there additional things we need to take care of when using HTTP2
when compared to HTTP1.1? More threads, different queue size, more file
handles etc?
Any pointers would be very much appreciated.
Kind regards,
Silvio
_______________________________________________
jetty-users mailing list
To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users
_______________________________________________
jetty-users mailing list
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users
_______________________________________________
jetty-users mailing list
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users