tag:highstakesswitzerland2.statuspage.io,2005:/historyHigh Stakes Switzerland Systems Monitoring Status - Incident History2023-12-19T06:25:54+01:00High Stakes Switzerland Systems Monitoringtag:highstakesswitzerland2.statuspage.io,2005:Incident/112865972022-09-27T03:00:00+02:002023-02-23T20:02:11+01:00Sentinel server offline<p><small>Sep <var data-var='date'>27</var>, <var data-var='time'>03:00</var> CEST</small><br><strong>Resolved</strong> - Server unexpectedly rebooted and network was unavailable.</p>tag:highstakesswitzerland2.statuspage.io,2005:Incident/112753412022-09-26T21:00:00+02:002023-02-23T20:02:11+01:00Injective AppHash error<p><small>Sep <var data-var='date'>26</var>, <var data-var='time'>21:00</var> CEST</small><br><strong>Resolved</strong> - Many validators crashed at block 15872289, probably due to a wasm error (post-mortem will follow).</p>tag:highstakesswitzerland2.statuspage.io,2005:Incident/110356212022-09-01T21:29:50+02:002023-02-23T20:02:12+01:00Injective data corruption<p><small>Sep <var data-var='date'> 1</var>, <var data-var='time'>21:29</var> CEST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Sep <var data-var='date'> 1</var>, <var data-var='time'>21:25</var> CEST</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Sep <var data-var='date'> 1</var>, <var data-var='time'>21:25</var> CEST</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:highstakesswitzerland2.statuspage.io,2005:Incident/107171002022-07-29T15:03:33+02:002022-07-29T15:03:34+02:00Server upgrade<p><small>Jul <var data-var='date'>29</var>, <var data-var='time'>15:03</var> CEST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Jul <var data-var='date'>29</var>, <var data-var='time'>14:53</var> CEST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jul <var data-var='date'>29</var>, <var data-var='time'>14:51</var> CEST</small><br><strong>Scheduled</strong> - We will be undergoing scheduled maintenance during this time.</p>tag:highstakesswitzerland2.statuspage.io,2005:Incident/103386352022-06-17T17:25:44+02:002022-06-17T17:25:44+02:00Website server maintenance<p><small>Jun <var data-var='date'>17</var>, <var data-var='time'>17:25</var> CEST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Jun <var data-var='date'>17</var>, <var data-var='time'>16:10</var> CEST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jun <var data-var='date'>17</var>, <var data-var='time'>16:09</var> CEST</small><br><strong>Scheduled</strong> - Some parts of the website will be redirected to a maintenance page for the time of the operations</p>tag:highstakesswitzerland2.statuspage.io,2005:Incident/100099312022-05-12T00:00:00+02:002023-02-23T20:02:12+01:00Terra death spiral<p><small>May <var data-var='date'>12</var>, <var data-var='time'>00:00</var> CEST</small><br><strong>Resolved</strong> - We are currently investigating this issue.</p>tag:highstakesswitzerland2.statuspage.io,2005:Incident/99473312022-05-08T23:30:00+02:002023-02-23T20:02:12+01:00Terra Oracle NOK price issue<p><small>May <var data-var='date'> 8</var>, <var data-var='time'>23:30</var> CEST</small><br><strong>Resolved</strong> - Following a breaking change at Currencylayer's api used by terra-oracle to retrieve price, NOK price was off for about 67 validators, counting as missed votes.</p>tag:highstakesswitzerland2.statuspage.io,2005:Incident/98442232022-04-25T18:00:23+02:002022-04-25T18:00:23+02:00Global security updates<p><small>Apr <var data-var='date'>25</var>, <var data-var='time'>18:00</var> CEST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Apr <var data-var='date'>24</var>, <var data-var='time'>12:00</var> CEST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Apr <var data-var='date'>24</var>, <var data-var='time'>11:57</var> CEST</small><br><strong>Scheduled</strong> - We will apply all pending security and operating system updates gradually on our datacenter</p>tag:highstakesswitzerland2.statuspage.io,2005:Incident/98098772022-04-19T13:26:39+02:002023-02-23T20:02:12+01:00Terra Disk Failure<p><small>Apr <var data-var='date'>19</var>, <var data-var='time'>13:26</var> CEST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Apr <var data-var='date'>19</var>, <var data-var='time'>10:13</var> CEST</small><br><strong>Investigating</strong> - NVMe drive critical failure during the night</p>tag:highstakesswitzerland2.statuspage.io,2005:Incident/96871882022-04-01T19:45:24+02:002022-04-01T19:45:24+02:00Band Protocol Migration<p><small>Apr <var data-var='date'> 1</var>, <var data-var='time'>19:45</var> CEST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Apr <var data-var='date'> 1</var>, <var data-var='time'>19:31</var> CEST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Apr <var data-var='date'> 1</var>, <var data-var='time'>19:00</var> CEST</small><br><strong>Scheduled</strong> - We will be migrating Band Protocol sentries to another datacenter to increase resilience</p>tag:highstakesswitzerland2.statuspage.io,2005:Incident/96849062022-04-01T13:49:10+02:002023-02-23T20:02:12+01:00Persistence Sentry high load<p><small>Apr <var data-var='date'> 1</var>, <var data-var='time'>13:49</var> CEST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Apr <var data-var='date'> 1</var>, <var data-var='time'>13:34</var> CEST</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:highstakesswitzerland2.statuspage.io,2005:Incident/96675692022-03-30T16:36:28+02:002023-02-23T20:02:12+01:00Sentinel high IOWait<p><small>Mar <var data-var='date'>30</var>, <var data-var='time'>16:36</var> CEST</small><br><strong>Resolved</strong> - An incident happened in the datacenter leading to the disconnection of the NFS drives during a few minutes. Everything is back to normal now.</p><p><small>Mar <var data-var='date'>30</var>, <var data-var='time'>16:33</var> CEST</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Mar <var data-var='date'>30</var>, <var data-var='time'>16:28</var> CEST</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:highstakesswitzerland2.statuspage.io,2005:Incident/96187012022-03-24T13:00:22+01:002022-03-24T13:00:22+01:00Terra servers upgrade<p><small>Mar <var data-var='date'>24</var>, <var data-var='time'>13:00</var> CET</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Mar <var data-var='date'>24</var>, <var data-var='time'>12:45</var> CET</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Mar <var data-var='date'>24</var>, <var data-var='time'>12:38</var> CET</small><br><strong>Scheduled</strong> - We will be undergoing scheduled maintenance during this time to upgrade hardware on the specified servers. Outage should not exceed 15 minutes.</p>tag:highstakesswitzerland2.statuspage.io,2005:Incident/96186742022-02-20T09:00:00+01:002023-02-23T20:02:12+01:00Injective chain down<p><small>Feb <var data-var='date'>20</var>, <var data-var='time'>09:00</var> CET</small><br><strong>Resolved</strong> - Injective team made an unplanned upgrade that resulted in the halt of the chain for ~2 hours.</p>tag:highstakesswitzerland2.statuspage.io,2005:Incident/96183872022-01-14T09:43:00+01:002023-02-23T20:02:12+01:00Band Protocol Oracle Outage<p><small>Jan <var data-var='date'>14</var>, <var data-var='time'>09:43</var> CET</small><br><strong>Resolved</strong> - AWS connectivity issues from 9:43AM to 10:29AM</p>