Epsilon support on-call duty + WAM/WDX tier 1 support

Description of Epsilon Support on-call duty

The primary responsibility of the person on duty is to ensure that critical and high priority issues reported through the Epsilon support channel (epsilon-support@weborama.com) are transmitted to the BigSea on-call team. The person on duty should also remain in contact with the user until the resolution of the issue.

Critical Priority Issue Examples	High Priority Issue Examples

Critical Priority Issue Examples	High Priority Issue Examples
A semantic AI application interface (e.g MoonFish) is not accessible or unavailable.	A semantic AI application's important feature (e.g create a new segment) is broken (or partially working) after a new release.
End Users lost all their created segments in their accounts	Many end users are unable to activate their existing segment
All batch processing jobs are failing.	Some of the batch processing Jobs are failing. For example, the streaming jobs of MoonFish taxonomies are in pending for more than 4 hours.

On-call Process

To remain within SLAs for critical and high priority issues, the person on duty should respond to the Epsilon user within one hour of receiving the request and provide an update every two hours until the issue has been resolved. Progression on the issue can be followed on the associated Jira ticket.

Contacts

On-call email: bigsea-oncall@weborama.com

Additional information about the current BigSea on-call person on duty can be seen here.

Epsilon On-call phone #: 0140151300

Mohamed: 0695286162
Ian : 0761158480
Guillaume: 0781622665
Narimen: 0766512935

Cédric: 0645176508

Schedule

https://docs.google.com/spreadsheets/d/1CfFTFovxl6x85wSUs3ow0KkD-rc017fe7YCwGSh0p-E/edit?pli=1#gid=2032019723

reference for SRE team planning = https://weborama.atlassian.net/wiki/spaces/BI/pages/557539

Description of WAM/WDX Support tier 1 on-call duty

The person in charge in Epsilon-On Call will be in charge also on answering WAM + WDX support tickets as a tier 1.

What does tier 1 mean?

A tier 1 is a first layer of support, who will during the weekly on-call give a 1st answer to the ticket, try to give an answer to the ticket, discuss with the concerned PO(s) to resolve the ticket and also create the JIRA ticket in the concerned projet.

This person will be paid for compensated for his intervention during the business days for Semantic Apps + WAM + WDX supports and for 7/7 days for Epsilon On-Call. Please find here more details about the conditions of an On-Call support: https://drive.google.com/file/d/1ukPVrZNqzAOx6H6Px7jAVWkguNQMuTHz/view?pli=1

How Product On-Call guy should handle WAM/WDX support?

The product on charge should receive an email or see a ticket in Zendesk when a ticket is created via “Tier 2 WAM“ group. You can visualize unsolved ticket via the following Zendesk view: https://adrime.zendesk.com/agent/filters/544610

First of all he or she should open the ticket, read it to understand it then choose a type and a priority for this ticket.

If the ticket is of type “incident“, the priority should be “urgent“ and “Product DMP On Call“ should be reached by email to ask them to look to the incident via production@weborama.com by describing it with details to enable the technical on-call guy to investigate quickly and correctly and then provide a first answer in the ticket for the client in the same business day. Here is the technical On-Call planning:

If the ticket is of type “problem“, the priority should be evaluated first.

If the priority is considered as “High“ the product on-call should provide a first answer to the client in the same business day order to let know that we are handling it. The product on-call guy should investigate within the help of the PO @Narimen Kadem for WAM and @Guillaume Pierre for WDX. If the product is not able to provide an answer the product on-call guy has to create Jira ticket in the corresponding product and reach @Simon Lenne for WAM investigation and @Ju for WDX investigation. If @Simon Lenne is absent @Ju could be then reached.
If the priority is considered as “Normal“ the product on-call should provide a first answer to the client in the same business week in order to let know that we are handling it. The product on-call guy should investigate first by himself and if he is not able to answer he could ask the help of the PO @Narimen Kadem for WAM and @Guillaume Pierre for WDX. If the product is not able to provide an answer the product on-call guy has to create Jira ticket in the corresponding product and assign it the corresponding PO who will be responsible on on adding the ticket to the product issue backlog or not and add it to a sprint when he will consider it is appropriate .
If the priority is considered as “Low“ the product on-call should provide a first answer to the client in the same business week in order to let know that we are handling it. The product on-call guy should investigate first by himself and if he is not able to answer he could ask the help of the PO @Narimen Kadem for WAM and @Guillaume Pierre for WDX. If the product is not able to provide an answer the product on-call guy has to create Jira ticket in the corresponding product and assign it the corresponding PO who will be responsible on adding the ticket to the product issue backlog or not add it to a sprint when he will consider it is appropriate.

Kibana dashboards =

Affectation per account = http://kibana-product.aub.ui.in.weborama.fr/app/kibana?#/discover?_g=(refreshInterval:(display:Off,pause:!f,value:0),time:(from:now-60d,mode:quick,to:now))&_a=(columns:!(_source),index:'logstash-*',interval:auto,query:(query_string:(analyze_wildcard:!t,query:'mdc.parameters.account_id:2510')),sort:!('@timestamp',desc))

affectation global =

http://kibana-product.service.aub.consul:5601/app/kibana#/visualize/edit/WAM-Account-Id?_g=(refreshInterval:(display:Off,pause:!f,value:0),time:(from:now-30d,mode:quick,to:now))&_a=(filters:!(('$state':(store:appState),meta:(alias:!n,disabled:!f,index:'logstash-',key:mdc.product,negate:!f,value:WAM),query:(match:(mdc.product:(query:WAM,type:phrase)))),('$state':(store:appState),meta:(alias:!n,disabled:!f,index:'logstash-',key:query,negate:!f,value:'%7B%22exists%22:%7B%22field%22:%22mdc.wam.assigned_audiences%22%7D%7D'),query:(exists:(field:mdc.wam.assigned_audiences)))),linked:!f,query:(query_string:(analyze_wildcard:!t,query:'*')),uiState:(),vis:(aggs:!((id:'1',params:(field:mdc.wam.uuids),schema:metric,type:sum),(id:'2',params:(field:mdc.parameters.account_id,order:desc,orderBy:'1',size:1000),schema:segment,type:terms)),listeners:(),params:(addLegend:!t,addTooltip:!t,isDonut:!f,shareYAxis:!t),type:pie))

IDsync =