วันศุกร์ที่ 9 พฤศจิกายน พ.ศ. 2561

Imbalanced data set

  • A problem in classification i.e. labels are discrete 
  • Binary class data set is imbalanced if  YES and NO classes (i.e., the labels of data points in the set) are not 50/50 (or 60/40) in terms of the number of data points. 
  •  A dataset is marginally imbalanced if one class is rare compared to the other class.
  • If the training set has a heavy imbalance where most labels are "man" rather than "woman," the model will generally predict "man" more frequently. This scenario has two problems: 1.Biased Decision Boundaries: The model's optimization algorithm shifts its internal boundary because minimizing errors on the majority class ("man") lowers the overall loss much faster than trying to get the minority class right. 2.Skewed Output Probabilities: The model learns the prior probabilities of the training data. Because "man" appears much more frequently, the model's output probabilities naturally favor it, requiring an unusually high amount of evidence for it to output "woman."
  • Solved by under sampling  (i.e. use all the smaller class and randomly select same number of majority class several times to make multiple data sets and then combine all classification results) to balance it this is best way for this without loosing information. If you use boosting you could alter the weights and balance data that way (https://www.researchgate.net/post/How_to_know_that_our_dataset_is_imbalance). อีกวิธีที่นิยมใช้แก้ปัญหาชื่อ SMOTE ซึ่งเป็นการสร้าง synthetic samples (i.e. oversampling) ขึ้นมา https://towardsdatascience.com/dealing-with-imbalanced-classes-in-machine-learning-d43d6fa19d2
  • สมมติให้ data set มี 2 classes Class A มีจำนวน 90 เปอร์เซ็นต์ Class B มีจำนวน 10 เปอร์เซ็นต์ ถ้าไม่แก้ปัญหา imbalance แล้วใช้วิธีแล้วใช้วิธี Random ให้ 90%  อยู่ใน A จะได้ accuracy 91% แต่ถ้าพยากรณ์ถูกหมดทั้ง A & B (อาจใช้over/undersampling ช่วย) จะได้ accuracy 100%





Recommendation systems

RS is a kind of information filtering (of items recommended to users).
Types:
  • Content-based RS : Recommendation of new items to users is performed by looking at (historic) item to (new) item (item is content) similarity. It uses item profile and user preference profile (i.e. items user liked) (Cf. "Handbook on ontologies")
  • Collaborative filtering RS : Recommendation of new items to user is performed based on item reviews (i.e. other users) within a community of the user. (Cf. "Handbook on ontologies") Collaborative filtering works by using the ratings provided by a community of users to recommend items for a specific user. There are two complementary approaches available, user-based or item-based collaborative filtering. User based collaborative filtering is where similar users are found and items recommended that these similar users also liked. Item-based collaborative filtering is where items are grouped if people rate them similarly then the items are recommended together.
  • Knowledge-based RS aka rule-based RS : RS of either above types that is supplimented by knowledge base e.g. to calculate user-item or item-item attribute similarity (i.e., similarity of attributes existing in both user and item) (https://medium.com/recombee-blog/recommender-systems-explained-d98e8221f468); The attributes may be in form of ontology-based RS (which has knowledge engineering bottleneck i.e. various problems in knowledge acquisition); this RS is aka rule-based RS in which the rule may be heuristic rules or ML-based classification rules like decision tree.
  • Hybrid RS is mix of the first two types.

User profiling methods:

  • Knowledge based approach : uses static models of users and dynamically match users to the closest model. Questionnaires and interviews are often employed to obtain this user knowledge. Once a model is selected for a user, specific domain knowledge (from ontology) for that user type can be applied to help describe the user.
  • Behavioral based approach : uses the user’s behaviour as a model, commonly using machine-learning techniques to discover useful patterns in the behaviour. Behavioural logging is employed to obtain the data necessary from which to extract patterns.

Common problem of RS is cold start problem (a state of a deployed RS having not enough info on item or user or community (in case of collaborative RS) to give recommendation). Cf. https://en.wikipedia.org/wiki/Cold_start_(computing)

วันพฤหัสบดีที่ 8 พฤศจิกายน พ.ศ. 2561

การอภิปรายผลของการพยากรณ์ด้วย machine learning


  • เมื่อได้โมเดลออกมาต้องลอง interpret โมเดล เช่น อ่าน decision tree ให้ได้
  • เมื่อได้ผลกาพยากรณ์ออกมาเป็นค่าแล้วต้องอธิบายสาเหตุว่าทำไมได้ค่าเท่านั้น เช่น พยากรณ์ว่าตู้ ATM นี้มี demand มากต้องอธิบายได้ว่าเพราะอะไร เพราะคนบริเวณนั้นยังไม่ใช้ cashless app. เพราะ ส่วนใหญ่เป็นผู้สูงอายุ เป็นต้น

วันพุธที่ 7 พฤศจิกายน พ.ศ. 2561

Data types in data mining : discrete, nominal, continuous, binary, ordinal













Src:  https://stats.stackexchange.com/questions/159902/is-nominal-ordinal-binary-for-quantitative-data-qualitative-data-or-both

Nominal เรียกอีกอย่างหนึ่งว่า categorical

Discretization = quantization = the process of constraining an input from a continuous or otherwise large set of values (such as the real numbers) to a discrete set (such as the integers). เช่น แรงดัน 2.5 ถึง 5V. ให้ถือเป็น 1; Opposite is "nominal to numeric" process.

A cardinal number tells "how many." Cardinal numbers are also known as "counting numbers," because they show quantity.

วันจันทร์ที่ 5 พฤศจิกายน พ.ศ. 2561

HTTP Strict Transport Security (HSTS)

HTTP Strict Transport Security (HSTS) allows web servers to declare that web browsers (or other complying user agents) should interact with it using only secure HTTPS connections,[1] and never via the insecure HTTP protocol.
When a web application issues HSTS Policy to user agents, user agents behave as follows:
1. Automatically turn any insecure links referencing the web application into secure links. (For instance, http://example.com/some/page/ will be modified to https://example.com/some/page/before accessing the server.)
2. If the security of the connection cannot be ensured (e.g. the server's TLS certificate is not trusted), the user agent must terminate the connection (and should not allow the user to access the application.
The HSTS Policy helps protect web application users against some passive (eavesdropping) and active network attacks. A man-in-the-middle attacker has a greatly reduced ability to intercept requests and responses between a user and a web application server.
HSTS can fix is SSL-stripping man-in-the-middle attacks, working by transparently converting a secure HTTPS connection into a plain HTTP connection. The user can see that the connection is insecure, but crucially there is no way of knowing whether the connection should be secure. Many websites do not use TLS/SSL, therefore there is no way of knowing (without prior knowledge) whether the use of plain HTTP is due to an attack, or simply because the website hasn't implemented TLS/SSL.
--wiki

วันอาทิตย์ที่ 14 ตุลาคม พ.ศ. 2561

วันพฤหัสบดีที่ 11 ตุลาคม พ.ศ. 2561

วันศุกร์ที่ 5 ตุลาคม พ.ศ. 2561

วันพุธที่ 26 กันยายน พ.ศ. 2561

vision mission strategy

### Vision
* **The aspirational future state.**
* A clear and inspiring picture of what the organization wants to achieve in the long term.
* Sets the overall direction and purpose.
* Example: "To be the world's leading provider of sustainable energy solutions."

### Mission
* **The company's purpose and reason for existence.**
* Defines the organization's core values, customers, and products or services.
* Bridges the gap between the vision and the strategies.
* Example: "To develop and deliver clean, affordable energy solutions that improve lives and protect the planet."

### Strategies
* **The roadmap to achieve the vision.**
* Specific plans and actions to realize the mission and vision.
* Involve resource allocation, competitive advantage, and market positioning.
* Example: "Expand into new renewable energy markets, invest in research and development for advanced technologies, and build strategic partnerships."

### The Interplay
* **Vision inspires mission.** The vision provides the overarching goal, motivating the definition of the mission.
* **Mission guides strategies.** The mission clarifies the organization's purpose, informing the development of effective strategies.
* **Strategies align with vision and mission.** Strategies should be directly linked to achieving the mission and ultimately realizing the vision.

**In essence:**
* The **vision** is the destination.
* The **mission** is the journey.
* **Strategies** are the steps along the way.

วันพุธที่ 19 กันยายน พ.ศ. 2561

A* algorithm






วันจันทร์ที่ 17 กันยายน พ.ศ. 2561

ปรัชญา (philosophy)

A system of thought based on or involving such study.

An underlying theory or set of ideas relating to a particular field of activity or to life as a whole.

วันอังคารที่ 11 กันยายน พ.ศ. 2561

Project consultant

https://m.pantip.com/topic/32250148?

ตามนี้ แต่ถ้าไม่ใช่พนักงานประจำ แค่รับทำเป็น project ก็เรียกว่า project consultant

วันเสาร์ที่ 8 กันยายน พ.ศ. 2561

Cross Browser Friendly Frameworks and tester

https://medium.com/@sarahelson81/how-to-make-a-cross-browser-compatible-website-40796eaeda05

https://www.lambdatest.com/index-new-landing.html?utm_source=Medium&utm_medium=Blog&utm_campaign=Blogpost&utm_term=indexnewlanding

Breakthrough vs. groundbreaking

As adjectives the difference between groundbreaking and breakthrough

is that groundbreaking is innovative; new, different; doing something that has never been done before while breakthrough is characterized by major progress or overcoming some obstacle.

วันศุกร์ที่ 24 สิงหาคม พ.ศ. 2561

infimum and supremum



วันพุธที่ 22 สิงหาคม พ.ศ. 2561

DVR

Local storage server of cctv.
Being replaced by cloud storage.
New cctv uses PoE and cloud storage.

NBIoT VS EMTC

EMTC is faster and good for mobile IoT.

AIS' NB-IoT