Speaking to the public

 

Customgpt VS Chatgpt

 CustomGPT utilizes ChatGPT for transfer learning in two primary ways:

1. Foundation Model: ChatGPT serves as the foundation for CustomGPT models. This means that the underlying architecture and knowledge base of ChatGPT are used as a starting point for training CustomGPT models.
2. Fine-Tuning: CustomGPT models are further trained on specific datasets relevant to their intended use cases. This process, known as fine-tuning, allows the models to learn and adapt to the unique characteristics of the target domain.

By combining ChatGPT's general-purpose capabilities with fine-tuning on specific datasets, CustomGPT can create highly specialized AI models that are tailored to perform well in various applications.

Neural Net's Learning rate adjustment

 One cycle policy (OCP)  is increasing and decreasing learning rate (i.e. adaptive learning rate) between upper & lower bounds to avoid slow training as much as possible.

https://medium.com/@varunsivamani/one-cycle-policy-a-deep-understanding-6d4d352ec7b1

https://medium.com/dsnet/the-1-cycle-policy-an-experiment-that-vanished-the-struggle-in-training-neural-nets-184417de23b9

SLA Tiers

Level 1 – 7/24/4 

• 7 days a week, 24 hours a day, reaction within 4 hours 
• 7×24 Help Desk 
• 7×24 Monitoring 
• Initial Phone-Based Diagnosis 
• Field Engineer on-site (if required) within 4 hours after customer contact 
• On-Site Spares Kit 
• Parts Replacement 
• Certified Experts 

Level 2 – 5/9/4 

• 8 hours a day, 5 working days per week, reaction within 4 hours 
• 8am-5pm Help Desk 
• 8am-5pm Monitoring 
• Initial Phone-Based Diagnosis 
• Field Engineer on-site (if required) within 4 hours after customer contact 
• On-Site Spares Kit 
• Parts Replacement 
• Certified Experts 

Level 3 – Next Day 

• Next business day response/parts 
• Initial Phone-Based Diagnosis  
• Field Engineer on-site (if required) the following business day after customer contact 
• On-Site Spares Kit 
• Parts Replacement 
• Certified Experts 

https://www.parkplacetechnologies.com/blog/what-does-24x7x4-support-mean-for-your-business/

Predicting Continuous Values with Transformers

Gemini:

Transformers can be effectively used to predict continuous values. While they were initially designed for natural language processing (NLP) tasks, their ability to capture long-range dependencies and complex patterns has made them versatile for various machine learning applications, including regression problems.

How Transformers Work for Regression

 * Input Encoding: The continuous values are typically encoded into a sequence of numerical representations, often using techniques like:

   * Embedding: For categorical variables

   * Normalization: For numerical variables

 * Positional Encoding: To provide the model with information about the relative position of each element in the sequence, positional encoding is added.

 * Transformer Layers: The input sequence is processed through multiple transformer layers, each consisting of:

   * Self-Attention: Captures relationships between different elements in the sequence.

   * Feed-forward Network: Applies a non-linear transformation to each element.

 * Output Layer: The final layer is a linear layer that maps the output of the transformer layers to a continuous value.

Advantages of Using Transformers for Regression

 * Long-range dependencies: Transformers can capture long-range dependencies, making them suitable for time series forecasting or tasks involving complex relationships between distant elements.

 * Flexibility: They can handle both sequential and non-sequential data, providing flexibility in various applications.

 * State-of-the-art performance: Transformers have achieved state-of-the-art results in many regression tasks, including time series forecasting and medical image analysis.

Key Considerations

 * Data preprocessing: Proper data preprocessing, including normalization and handling missing values, is crucial for optimal performance.

 * Hyperparameter tuning: Experimenting with different hyperparameters, such as the number of layers, attention heads, and hidden dimensions, can help improve results.

 * Computational resources: Training large transformer models can be computationally expensive, requiring significant GPU resources.

In conclusion, Transformers are a powerful tool for predicting continuous values, especially in tasks involving complex relationships and long-range dependencies. By understanding their architecture and key considerations, you can effectively leverage them for your regression problems.

Claude:

Transformers can be used to predict continuous values. This is often referred to as "regression" in machine learning terminology. Here's a brief overview:

1. Output layer modification: Instead of using a softmax layer for classification, you'd use a linear layer to output a single continuous value.

2. Loss function: You'd typically use mean squared error (MSE) or mean absolute error (MAE) as the loss function, rather than cross-entropy used in classification tasks.

3. Architecture: The core Transformer architecture (self-attention, feed-forward layers, etc.) remains largely the same.

4. Applications: This approach has been used successfully in various domains, including:

   - Time series forecasting

   - Financial predictions

   - Scientific modeling (e.g., predicting physical properties)

5. Variants: Some models like Temporal Fusion Transformers (TFT) are specifically designed for time series regression tasks.

Example:

https://medium.com/@Matthew_Frank/stock-price-prediction-using-transformers-2d84341ff213#:~:text=A%20Global%20Average%20Pooling%20layer,values%2C%20such%20as%20stock%20prices.

Examples of transformer models:

1. BERT (Bidirectional Encoder Representations from Transformers)  

• A powerful language model that can understand the context of a word based on its surrounding words.  
• Widely used for various NLP tasks like text classification, question answering, and text generation.  

2. GPT-3 (Generative Pre-trained Transformer 3)  

• A state-of-the-art language model capable of generating human-quality text.  
• Can be used for tasks like writing different kinds of creative content, translating languages, and answering questions in an informative way.  

3. DistilBERT

• A smaller, faster version of BERT, trained using knowledge distillation.  
• Maintains most of BERT's performance while being more efficient.  
• Ideal for resource-constrained environments or real-time applications.  

4. RoBERTa (Robustly Optimized BERT Pretraining Approach)  

• An improved version of BERT, trained on a larger dataset and with more aggressive hyperparameters.  
• Often outperforms BERT on various NLP benchmarks.

5. T5 (Text-To-Text Transfer Transformer)

• A unified framework for different text-to-text tasks, including translation, summarization, and question answering.  
• Can be fine-tuned on specific tasks with minimal effort.

6. XLNet

• A generalized autoregressive pretraining method that outperforms BERT on many NLP benchmarks.  
• Captures bidirectional context while avoiding the limitations of masked language modeling.  

7. BART (Bidirectional and Auto-Regressive Transformers)

• A model designed for both generative and discriminative tasks.
• Can be used for tasks like text summarization, question answering, and text generation.  

Key Advantages of Transformer Models:

• Strong performance: Transformer models consistently achieve state-of-the-art results on a wide range of NLP tasks.  
• Flexibility: They can be adapted to various tasks with minimal modifications.
• Scalability: They can be scaled to handle large datasets and complex tasks.  
• Interpretability: While still a challenge, techniques are being developed to better understand how transformer models work.

Benchmark & Baseline

Benchmarking involves measuring and comparing the performance of systems, models, or processes against a standard or across different systems. This process helps to evaluate how well a model or system performs in relation to known standards or to other models. In machine learning, the benchmark is usually set by the performance of the baseline or other leading models.

Benchmarking uses baselines: The baseline serves as the initial point of comparison in benchmarking. When evaluating a model or system, the baseline provides the first performance standard. If a model performs better than the baseline, it’s an indicator that the model has some value, and further benchmarking against other models can help assess its true effectiveness.

Baseline establishes expectations: Without a baseline, benchmarking would lack a clear starting point. By defining what "acceptable" or "expected" performance looks like, the baseline enables meaningful benchmarking comparisons.

Normalized RMSE & Normalized MAE

To compare model performance across different datasets by scaling the error metrics, you can normalize both RMSE and MAE:

Normalized RMSE:

• using the mean of data set: $NRMSE=\frac{RMSE}{\overline{y}}$ (i.e., Coefficient of Variation of RMSE)

• using the difference between maximum and minimum in data set: $NRMSE=\frac{RMSE}{y_{max}-y_{min}}$,

• using the standard deviation of data set: $NRMSE=\frac{RMSE}{\sigma }$, or

• using the interquartile range of data set; $NRMSE=\frac{RMSE}{Q1-Q3}$, i.e. the difference between 25th and 75th percentile, of observations.

https://www.marinedatascience.co/blog/2019/01/07/normalizing-the-rmse/

Normalized MAE: 

• aka. Coefficient of Variation of MAE; Coefficient of Variation (CV) =  the ratio of the standard deviation ${\displaystyle \sigma }$ to the mean ${\displaystyle \mu }$, ${\displaystyle CV=\frac{\sigma }{\mu }.}$

• using the mean of data set: $N\mathrm{MA}E=\frac{\mathrm{MA}E}{\overline{y}}$ 

https://d2uars7xkdmztq.cloudfront.net/app_resources/38997/documentation/135539_en.pdf