
Pruebas A / B
La importancia de las pruebas A / B y cómo las uso en mi trabajo
¿Qué aprendà sobre las pruebas A / B?
Las pruebas A / B me han permitido realizar cambios cuidadosos en sus experiencias de usuario mientras recopila datos sobre los resultados. Esto me ayuda a construir hipótesis y aprender por qué ciertos elementos de la experiencia de mi cliente impactan en el comportamiento del usuario. De otra manera, cuando les doy recomendaciones de nuevos diseños, corrección de diseños actuales, esta es una forma de respaldar mis recomendaciones o, a veces, aprender por qué ciertas cosas no funcionan para ciertos clientes.
La razón por la que muchas prácticas comunes funcionan en algunos clientes y en otros no es por el tipo de audiencia, el mercado, el tipo de negocio, el tráfico actual y muchos otros elementos.
Cada negocio es único, será imposible predecir cómo afectarán los cambios al sitio sin probarlos.
​
¿Cómo consigo que los clientes se unan para probar todas las recomendaciones y no implementarlas de inmediato?
Antes de presentar mis nuevas ideas, siempre les doy una introducción rápida o un repaso a las mejores prácticas de A / B, y les explico la importancia de realizar esas pruebas antes de implementarlas.
¿Las pruebas A / B son adecuadas para mi cliente?
Si el tráfico y las transacciones son bajos (menos de 1000 transacciones por mes) en el sitio, deberÃamos optar por otros métodos además de las pruebas A / B.
En cambio, considerarÃa métodos de investigación cualitativa para recopilar conocimientos de optimización como:
Encuesta al consumidor
Prueba de usuario
Evaluaciones de usabilidad

Cosas que debe saber antes de las pruebas A / B:
En promedio, el 30% de las pruebas A / B tienen éxito. Esto significa que el 70% de los resultados no tendrán ningún impacto o tendrán un impacto negativo en el sitio.
Los resultados negativos de la prueba no son fallas, y cuando obtengo un resultado negativo, solo confirma la importancia de esa prueba y me ayuda a elegir los siguientes pasos y qué evitar en el futuro.
Diagram

Research
We asked 7 of our customers the following questions in order to understand their problem as well as their current process when infractions are committed on-site.
1. Worker Status Check Process:
-
Company K: Workers are checked during initial site induction and tracked through a register. The dashboard is not considered accurate.
-
Company A: Timesheets are done daily, but worker status is not rigorously checked.
-
Company N: The site team is responsible for checking worker profiles.
2. Flagging at Division and Project Level:
-
Company K: Flags are applied company-wide.
-
Company A: Flags should indicate if someone is carded off a project but raises legality concerns.
-
Company N: Flags can be applied at both division and project levels.
-
Company R: Flags should apply to both division and project levels.
3. Red Flag Significance:
-
In all companies (K, A, N, R), a red flag signifies severe non-conformance, often due to rule-breaking or safety violations.
4. Actions for Red Flagged Workers:
-
Blocking access to check-in and qualifications is desired in all companies (K, A, N, R).
-
In Company A, if subcontractors invite red-flagged workers, they should be informed of the situation.
5. Reasons for Removing Red Flags:
-
Company K: Red flags can be removed based on time-weighted assessments and proof of additional training.
-
Company R: Red flags may be removed after investigations, clearance, or retraining.
6. Reasons for Red and Yellow Flags:
-
Red flags are typically associated with major safety violations and rule-breaking in all companies.
-
Company A mentioned unsafe behavior, stealing, drinking prior to work, and disruptive behavior as reasons.
-
Company R does not use yellow flags; workers are either compliant or not.
7. Duration of Red Flags:
-
Company N: Red flags prevent a worker from working for the company for two years. Re-entry may require evidence of behavior improvement through training or other measures.
8. Access to Other Projects/Divisions:
-
In Company A, access to other projects/divisions may be allowed if the subcontractor invites the red-flagged worker.
In summary, all companies recognize the importance of flagging for safety and compliance purposes. Red flags are uniformly associated with severe non-conformance or safety violations, leading to actions such as blocking access and qualification suspension. The duration of red flags varies, with some companies requiring evidence of improvement for re-entry. Legal and logistical issues concerning flagging at division and project levels are mentioned by Company A. Company R does not employ yellow flags, emphasizing a binary compliant/non-compliant system.
​
After prototyping the solution we tested the feature with 7 of our customers.
Here's an analysis of the provided data on feature testing and customer feedback related to worker flagging:
​
N Feedback:
-
Expectations for Worker Flagging: N expects complete access removal for flagged workers for two years.
-
Removing Worker Invitation: N doesn't mind removing worker invitations but is concerned about workers rejoining with different email addresses.
-
Infraction Feedback: N is comfortable with broad infraction descriptions.
-
Flag Creation/Removal Access: N suggests all admins should have access.
-
Current Flagging Process: N uses red and yellow flags in Excel files and removes worker invitations.
-
Notifications: N wants to notify admins, subcontractors, and workers.
-
General Feedback: N finds the feature exciting and helpful for tracking.
K Feedback:
-
Expectations for Worker Flagging: Removal at the account level for red flags, no access; yellow flags are warnings.
-
Removing Worker Invitation: K doesn't mind removing invitations but wants tracking.
-
Infraction Feedback: K is satisfied with the provided infraction descriptions.
-
Flag Creation/Removal Access: Likely project admins, as long as creators are visible.
-
Current Flagging Process: K is not currently flagging and sees value in yellow flags for tracking smaller warnings.
-
Notifications: K wants to notify admins, subcontractors, and workers.
-
General Feedback: K believes the feature aligns with their needs and processes.
R Feedback:
-
Expectations for Worker Flagging: Initially uninterested in yellow flags but finds value after understanding the purpose.
-
Removing Worker Invitation: Prefers worker removal from the project; not concerned about inviting again.
-
Infraction Feedback: Satisfied with infractions due to GDPR restrictions.
-
Flag Creation/Removal Access: Admins should have access; managers should not.
-
Current Flagging Process: Uninvites workers from the site.
-
Notifications: R wants to notify all admins, managers, subcontractors, and workers.
-
General Feedback: R appreciates the feature's usefulness and suggests enhancements, such as a flag in the worker list and showing the expiry date for yellow flags.
MCy Feedback:
-
Current Infraction Recording: MCy uses Orgamy for safety warnings and violations, but it has issues and doesn't help with flagging.
-
Interpretation of Red and Yellow Flags: Red flags indicate documented, repeated infractions; yellow flags imply safety situations with less severe consequences.
-
Worker Sanctions: Safety situations are typically at the project level, but severe infractions can extend to the region or account level, lasting one year.
-
Additional Infractions: MCy suggests removing rather than adding more infractions due to data entry concerns.
-
Flag Creation/Removal Access: General administrators and occasionally project managers.
-
Invitation Removal: MCy is fine with removing invitations for red-flagged workers.
-
Worker Qualification: Red-flagged workers should go unqualified or be removed from the list.
-
Overall Feedback: MCy sees the value in the feature, even though they may not use it frequently. They suggest making categories smaller.
In summary, the feedback from different companies (N, K, R, and MCy) generally highlights a positive response to the worker flagging feature. They appreciate the clarity of the process and desire notifications for various stakeholders. The use of red and yellow flags for different severity levels is common, with red flags typically leading to the removal of access and qualification.
Prototype:​



