1. Only the paired groups can be used to perform the paired test.
   
2. Each score is sampled independently and randomly. 
3. The scores are normally distributed within each of the two groups.
4. The variance in each of the groups is equal.

Since students from batch of 2009 and 2015 are not same, we should use Independent T-test rather than Paired T-test.

Sample I Data

Sample II Data

t.test(batch2009, batch2015, var.equal=FALSE)

When the var.equal argument is set to FALSE in the above syntax, it runs Welch's two sample t-test.

Welch Two Sample T-test

Since p-value is greater than 0.05, it means we fail to reject the null hypothesis. It implies there is no significant difference between the mean number of enrollment in statistics class in batch 2009 and batch 2015.

Welch's t-test performs better than Student's t-test whenever sample sizes and variances are unequal between groups, and gives the same result when sample sizes and variances are equal.

temp = rbind(data.frame(x=batch2009, group=1), data.frame(x=batch2015, group=2))
temp$group = as.factor(temp$group)
library(car)
leveneTest(temp$x, group = temp$group, center = mean)

xhat1 <- mean(batch2009)
xhat2 <- mean(batch2015)
s1 <- sd(batch2009)
s2 <- sd(batch2015)
n1 <- length(batch2009)
n2 <- length(batch2015)
diff_in_means <- xhat1 - xhat2
SE_diff_mean <- sqrt(s1^2/n1+s2^2/n2)
t_stat <- diff_in_means/SE_diff_mean
t_stat
pvalue = 2* pt(t_stat, df=n1+n2-2)
pvalue

ods select Quantiles;
proc univariate data = sashelp.shoes;
var sales;
class region;
run;

ods trace on;
proc univariate data = sashelp.shoes;
var sales;
run;
ods trace off;

ods output Quantiles = temp;
proc univariate data = sashelp.shoes;
var sales;
class region;
run;
ods output close;

ods output extremeobs = outlier;
proc univariate data = sashelp.shoes;
var sales;
class region;
run;
ods output close;

1. Plot Histogram and see the distribution
2. Calculate Skewness
3. Normality Tests

proc univariate data=sashelp.shoes NOPRINT;
var sales;
HISTOGRAM / NORMAL (COLOR=RED);
run;

Skewness

Positive Skewness : If skewness > 0, data is positively skewed. Another way to see positive skewness : Mean is greater than median and median is greater than mode.

Negative Skewness : If skewness < 0, data is negatively skewed. Another way to see negative skewness : Mean is less than median and median is less  than mode.

1. If skewness < −1 or > +1, the distribution is highly skewed.
2. If skewness is between −1 and −0.5 or between 0.5 and +1, the distribution is moderately skewed.
3. If skewness > −0.5 and  <  0.5, the distribution is approximately symmetric or normal.

Skewness and Normality

ods select TestsforNormality;
proc univariate data = sashelp.shoes normal;
var sales;
run;

Tests for Normality

If p-value > 0.05, data is normal. In the example above, p-value is less than 0.05, it means data is not normal.

proc univariate data = sashelp.shoes noprint;
var sales;
output out = temp
pctlpts = 10 to 100 by 10 pctlpre = p_;
run;

proc univariate data = sashelp.shoes noprint;
var sales;
output out = temp
pctlpts = 97.5,99.5 pctlpre = p_;
run; 

ods select winsorizedmeans;
ods output winsorizedmeans=means;
proc univariate winsorized = 0.2 data=sashelp.shoes;
var sales;
run;

Winsorized Means

ods select trimmedmeans;
ods output trimmedmeans=means;
proc univariate trimmed = 0.2 data=sashelp.shoes;
var sales;
run;

ods select TestsForLocation;
proc univariate data=sashelp.shoes;
var sales;
run;

Ttest with PROC Univariate

1. Histogram
2. Box Plot
3. Normal Probability Plot

proc univariate data=sashelp.shoes PLOT;
var sales;
run;

Looking ahead some number of rows or periods.

Lead Value

data temp;
set example;
x + 1;
run;

proc sort data = temp;
by descending x;
run;

data temp2;
set temp;
lead_value = lag(value);
run;

proc sort data = temp2;
by x;
run;

data temp;
_N_+1;
if _N_ <= k then do;
set example point=_N_;
lead_value = value;
end;
else lead_value = .;
set example nobs=k;
proc print noobs;
run;

data temp;
set example;
by id;
if first.id then x =1;
else x + 1;
run;

proc sort data = temp;
by id descending x;
run;

data temp2;
set temp;
by id descending x;
l = lag(x);
lead_val = lag(value);
if first.id then do;
l = .;
lead_val = .;
end;
run;

proc sort data = temp2;
by id x;
run;

Lead Variables

data example;
set example;
by id;
if first.id then N = 1;
else N + 1;
run;

proc sql;
select a.* , b.value as lag_value
from example a left join example b
on a.id = b.id and a.N + 1 = b.N;
quit;

data example;
 set example;
 lagid = lag(id);
run;  

data leads;
 _n_ + 1;
 if _n_ <= n then do;
 set example point=_n_;
 if lagid = id then do;
 y = value;
 end;
 end;
 else y = .;
 set example nobs = n;
 by id;
 drop lagid;
 proc print noobs;
 run;

proc expand data= example out= t2;
convert value = lead_value /transformout = (lead 1);
by id;
run;

The names of dplyr functions are similar to SQL commands such as select() for selecting variables, group_by() - group data by grouping variable, join() - joining two data sets. Also includes inner_join() and left_join(). It also supports sub queries for which SQL was popular for.

install.packages("dplyr")

library(dplyr) 

Example 11 : Rename Variables

The rename function can be used to rename variables.

In the following code, we are renaming 'Index' variable to 'Index1'.

mydata6 = rename(mydata, Index1=Index)

Output

filter( ) Function

It is used to subset data with matching logical conditions.

filter() syntax : filter(data , ....)
data : Data Frame
.... : Logical Condition

Example 12 : Filter Rows

mydata10 = filter(mydata6, grepl("Ar", State))

summarise() syntax : summarise(data , ....)
data : Data Frame
..... : Summary Functions such as mean, median etc

summarise(mydata, Y2015_mean = mean(Y2015), Y2015_med=median(Y2015))

Output

summarise_at(mydata, vars(Y2005, Y2006), funs(n(), mean, median))

Output

summarise_at(mydata, vars(Y2011, Y2012),
funs(n(), missing = sum(is.na(.)), mean(., na.rm = TRUE), median(.,na.rm = TRUE)))

Summarize : Output

Example 21 : Summarize all Numeric Variables

summarise_all(mydata["Index"], funs(nlevels(.), sum(is.na(.))))

arrange(data_frame, variable(s)_to_sort)
or
data_frame %>% arrange(variable(s)_to_sort)

arrange(mydata, Index, Y2011)

It allows you to write sub-queries like we do it in sql.

Note : All the functions in dplyr package can be used without the pipe operator. The question arises "Why to use pipe operator %>%". The answer is it lets to wrap multiple functions together with the use of  %>%.

Syntax :

filter(data_frame, variable == value)
or
data_frame %>% filter(variable == value)

The %>% is NOT restricted to filter function. It can be used with any function. 

Example :

The code below demonstrates the usage of pipe %>% operator. In this example, we are selecting 10 random observations of two variables "Index""State" from the data frame "mydata".

dt = sample_n(select(mydata, Index, State),10)
or 
dt = mydata %>% select(Index, State) %>% sample_n(10)

Output

group_by() function :

Use : Group data by categorical variable

Syntax :

group_by(data, variables)
or
data %>% group_by(variables)

Example 24 : Summarise Data by Categorical Variable

We are calculating count and mean of variables Y2011 and Y2012 by variable Index.

t = summarise_at(group_by(mydata, Index), vars(Y2011, Y2012), funs(n(), mean(., na.rm = TRUE)))

The above code can also be written like

t = mydata %>% group_by(Index) %>%
  summarise_at(vars(Y2011:Y2015), funs(n(), mean(., na.rm = TRUE)))

do() function :

Use : Compute within groups

Syntax :

do(data_frame, expressions_to_apply_to_each_group)

Note : The dot (.) is required to refer to a data frame.

Example 25 : Filter Data within a Categorical Variable

Suppose you need to pull top 2 rows from 'A', 'C' and 'I' categories of variable Index. 

t = mydata %>% filter(Index %in% c("A", "C","I")) %>% group_by(Index) %>%
  do(head( . , 2))

Output : do() function

Example 26 : Selecting 3rd Maximum Value by Categorical Variable

We are calculating third maximum value of variable Y2015 by variable Index. The following code first selects only two variables Index and Y2015. Then it filters the variable Index with 'A', 'C' and 'I' and then it groups the same variable and sorts the variable Y2015 in descending order. At last, it selects the third row.

t = mydata %>% select(Index, Y2015) %>%
  filter(Index %in% c("A", "C","I")) %>%
  group_by(Index) %>%
  do(arrange(.,desc(Y2015))) %>%  slice(3)

The slice() function is used to select rows by position.

Output

Using Window Functions

Like SQL, dplyr uses window functions that are used to subset data within a group. It returns a vector of values. We could use min_rank() function that calculates rank in the preceding example,

t = mydata %>% select(Index, Y2015) %>%
  filter(Index %in% c("A", "C","I")) %>%
  group_by(Index) %>%
  filter(min_rank(desc(Y2015)) == 3)

Example 27 : Summarize, Group and Sort Together 

In this case, we are computing mean of variables Y2014 and Y2015 by variable Index. Then sort the result by calculated mean variable Y2015.

t = mydata %>%
  group_by(Index)%>%
  summarise(Mean_2014 = mean(Y2014, na.rm=TRUE),
            Mean_2015 = mean(Y2015, na.rm=TRUE)) %>%
  arrange(desc(Mean_2015))

mutate() function :

Use : Creates new variables

Syntax :

mutate(data_frame, expression(s) )
or
data_frame %>% mutate(expression(s))

Output

Output

out = mydata %>% group_by(Index) %>% filter(min_rank(desc(Y2015)) == 1) %>%
  select(Index, Y2015)

out2 = mydata %>% group_by(Index) %>% mutate(Total=cumsum(Y2015)) %>%
select(Index, Y2015, Total)

join() function :

Use : Join two datasets

Syntax :

inner_join(x, y, by = )
left_join(x, y, by = )
right_join(x, y, by = )
full_join(x, y, by = )
semi_join(x, y, by = )
anti_join(x, y, by = )

x, y - datasets (or tables) to merge / join
by - common variable (primary key) to join by.

Example 33 : Common rows in both the tables

Let's create two data frames say df1 and df2.

df1 <- data.frame(ID = c(1, 2, 3, 4, 5),
                  w = c('a', 'b', 'c', 'd', 'e'),
                  x = c(1, 1, 0, 0, 1),
                  y=rnorm(5),
                  z=letters[1:5])

df2 <- data.frame(ID = c(1, 7, 3, 6, 8),
                  a = c('z', 'b', 'k', 'd', 'l'),
                  b = c(1, 2, 3, 0, 4),
                  c =rnorm(5),
                  d =letters[2:6])

INNER JOIN returns rows when there is a match in both tables. In this example, we are merging df1 and df2 with ID as common variable (primary key).

df3 = inner_join(df1, df2, by = "ID")

Output : INNER JOIN

If the primary key does not have same name in both the tables, try the following way:

inner_join(df1, df2, by = c("ID"="ID1"))

Example 34 : Applying LEFT JOIN

LEFT JOIN : It returns all rows from the left table, even if there are no matches in the right table.

left_join(df1, df2, by = "ID")

Output : LEFT JOIN

setdiff(first, second)

if_else(condition, true, false, missing = NULL)

Output

df = mydata %>%
  rowwise() %>% mutate(Max= max(Y2012:Y2015)) %>%
  select(Y2012:Y2015,Max)

Output

df1=data.frame(ID = 1:6,  x=letters[1:6])
df2=data.frame(ID = 7:12, x=letters[7:12])

Input Datasets

The bind_rows() function combine two datasets with rows. So combined dataset would contain 12 rows (6+6) and 2 columns.

xy = bind_rows(df1,df2)

xy = rbind(df1,df2)

The bind_cols() function combine two datasets with columns. So combined dataset would contain 4 columns and 6 rows.

xy = bind_cols(x,y)
or
xy = cbind(x,y)

The output is shown below-

cbind Output

Example 41 : Calculate Percentile Values

The quantile() function is used to determine Nth percentile value. In this example, we are computing percentile values by variable Index.

mydata %>% group_by(Index) %>%
  summarise(Pecentile_25=quantile(Y2015, probs=0.25),
            Pecentile_50=quantile(Y2015, probs=0.5),
            Pecentile_75=quantile(Y2015, probs=0.75),
            Pecentile_99=quantile(Y2015, probs=0.99))

The ntile() function is used to divide the data into N bins.

x= data.frame(N= 1:10)
x = mutate(x, pos = ntile(x$N,5))

Example 42 : Automate Model Building

This example explains the advanced usage of do() function. In this example, we are building linear regression model for each level of a categorical variable. There are 3 levels in variable cyl of dataset mtcars.

length(unique(mtcars$cyl))

Result : 3

by_cyl <- group_by(mtcars, cyl)
models <- by_cyl %>% do(mod = lm(mpg ~ disp, data = .))
summarise(models, rsq = summary(mod)$r.squared)
models %>% do(data.frame(
  var = names(coef(.$mod)),
  coef(summary(.$mod)))
)

Output : R-Squared Values

1. Customers targeted in Jan 2015 for the home loan campaign–whether the customers have taken the loan from Feb 2015 to April 2015
2. Customers targeted in Feb 2015 for the home loan campaign–whether the customers have taken the loan converted from March 2015 to May 2015
3. Customers targeted in March 2015 for the home loan campaign–whether the customers have taken the loan from April 2015 to June 2015

options validvarname=any;
data temp2;
set temp;
rename var1 = '1variable'n;
run;

options VALIDMEMNAME=EXTEND;
proc print data= 'price data'n;
run;

x <- c(4,5,6,7)
y <- c(2,3)
x*y

1. Vector
2. Matrix
3. Array
4. List
5. Data frame
6. Factor

x = c(1:5)
y = c("m","f","f","m","f")

z=cbind(x,y)

z = rbind(x,y)

df = data.frame(x = c(1:4), y = c("m","f","f","m"))
df2 = data.frame(x = c(5:8))

library(dplyr)
combdf = bind_rows(df,df2)

A variable name such as .1var is not valid. But .var1 is valid.

if else repeat while function for in next break
TRUE FALSE NULL Inf NaN NA NA_integer_ NA_real_ NA_complex_ NA_character_

df=data.frame(x=c(1:6), y=c(1,2,4,6,8,12))

(df$x + df$y) + (df$x - df$y)

with(df, (x+y) + (x-y))

library(dplyr)
df %>% mutate((x+y) + (x-y))

df = data.frame(var1=factor(c(1,2,1,2,1,2)), var2=c(10:15))
with(df, by(df, var1, function(x) mean(x$var2)))

library(dplyr)
df %>% group_by(var1)%>% summarise(mean(var2))

data = read.table(text="
X Y Z
NA 1 5
3 NA 2
", header=TRUE)

library(dplyr)
data %>% mutate(var=coalesce(X,Y,Z))

dt1 = read.table(text="
X Y Z
7 NA 5
2 4 5
", header=TRUE)

dt1$var = apply(dt1,1, function(x) max(x,na.rm = TRUE)) 

dt2 = read.table(text="
A B C
8 0 0
6 0 5
", header=TRUE)

apply(dt2,1, function(x) sum(x==0))

ifelse(df$var1==NA, 0,1)

Output

x = 3
mult <- function(j)
{
  x <<- j * 2
  return(x)
}
mult(2)
x

a <- factor(c(5, 6, 7, 7, 5))
a1 = as.numeric(as.character(a))

a = "Deepanshu"
b = "Bhalla"
paste(a, b)

paste(a, b, sep=",") returns "Deepanshu,Bhalla"

x = "AXZ2016"
substr(x,1,3)

dt2 = read.table(text="
var
Sandy,Jones
Dave,Jon,Jhonson
", header=TRUE)

library(stringr)
dt2$var2 = word(dt2$var, -1, sep = ",")

a = " David Banes "
trimws(a)

rand = runif(100, min = 1, max = 100)

df1=data.frame(ID=c(1:5), Score=runif(5,50,100))
df2=data.frame(ID=c(3,5,7:9), Score2=runif(5,1,100))
comb = merge(df1, df2, by ="ID", all.x = TRUE)

library(sqldf)
comb = sqldf('select df1.*, df2.* from df1 left join df2 on df1.ID = df2.ID')

library(dplyr)
comb = left_join(df1, df2, by = "ID")

comb = merge(df1,df2,by=NULL)

library(sqldf)
comb2 = sqldf('select * from df1 join df2 ')

library(dplyr)
comb = intersect(df1,df2)

library(sqldf)
comb2 = sqldf('select * from df1 intersect select * from df2 ')

Output : Intersection with R

start.time <- Sys.time()
runif(5555,1,1000)
end.time <- Sys.time()
end.time - start.time

library(tictoc)
tic()
runif(5555,1,1000)
toc()

# Load data
library(nycflights13)
data(flights)
df = setDT(flights)

# Load required packages
library(tictoc)
library(dplyr)
library(data.table)

# Using data.table package
tic()
df[arr_delay > 30 & dest == "IAH",
   .(avg = mean(arr_delay),
     size = .N),
   by = carrier]
toc()

# Using dplyr package
tic()
flights %>% filter(arr_delay > 30 & dest == "IAH") %>%
  group_by(carrier) %>% summarise(avg = mean(arr_delay), size = n())
toc()

library(data.table)
yyy = fread("C:\\Users\\Dave\\Example.csv", header = TRUE)

rm(list=ls())

1. Bubble Sort
2. Selection Sort
3. Merge Sort
4. Quick Sort
5. Bucket Sort

df = subset(mydata, select = -c(x,y,z))

library(dplyr)
test1 = distinct(data, y, .keep_all= TRUE)

dates <- as.Date(c("2015-09-02", "2016-09-05"))

difftime(dates[2], dates[1], units = "hours")
difftime(dates[2], dates[1], units = "days")
floor(difftime(dates[2], dates[1], units = "weeks"))
floor(difftime(dates[2], dates[1], units = "days")/365)

library(lubridate)
interval(dates[1], dates[2]) %/% hours(1)
interval(dates[1], dates[2]) %/% days(1)
interval(dates[1], dates[2]) %/% weeks(1)
interval(dates[1], dates[2]) %/% months(1)
interval(dates[1], dates[2]) %/% years(1)

mydate <- as.Date("2015-09-02")
mydate + months(3)

mydate <- as.POSIXlt("2015-09-27 12:02:14")
library(lubridate)
date(mydate) # Extracting date part
format(mydate, format="%H:%M:%S") # Extracting time part

set.seed(1234)
x = sample(1:50, 10)  
x

linreg1 <- lm(y ~ x1*x2, data=mydata)

dt = sort(sample(nrow(mydata), nrow(mydata)*.7))
train<-mydata[dt,]
val<-mydata[-dt,]

1. inputdata : Specify the name of the dataset in which open-ended comments exist
2. var : Specify the name of the variable which contains comments
3. outdata : Specify the name you want to assign to the output dataset

The Australian Security Agency designed a terror attack system that let their citizens a clearer idea of whether they should be alert or alarmed. It classifies threats into five levels – Not Expected, Possible, Probable, Expected and Certain.

Target / Dependent Variable - Whether a person is terrorist or not
Predictors / Independent Variable - 80 Variables. Some of the variables are listed below -

Number of Events  :  Data from just seven known terrorists.
Number of Non-Events : 100,000 users were selected at random

A false positive rate of 0.18 percent across 55 million people would mean 99,000 innocents mislabelled as "terrorists"

The highest rated target according to this machine learning was Ahmad Zaidan, Al-Jazeera's long-time bureau chief in Islamabad.

1. Event Rate : The main issue of the model is that they used a very few events (7 terrorists) to train the model. Machine learning algorithms require more events than classical statistical techniques. 
2. Unstructured Data : Huge amount of data but unstructured
3. Collaboration between Countries : Official data sharing security pact
4. Implementation : It is very dangerous to implement the model and kill someone after blindly following results from the model.

1. array values y1-y6 : Lists all the variables for calculating max value
2. max() function calculates maximum value of all the variables listed in step 1 across rows
3. whichn() function returns the column index number of the matching value. In this case, it is searching the maximum value across rows and returns the column position in the listed variables. For example, it returns 6 in row 1 as 87 is the maximum value in row 1 and it is placed at 6th column of y1-y6
4. vname() function returns the variable name. In this case, it calculates variable name of the largest value.

1. array values[*] y1-y6  -  Specify all the variables from which you want to calculate top 3 variables
2. array large[3] - Top 3 large values
3. array names[3] $32 - Names of top 3 variables
4. do i = 1 to dim(large) - 3 iterations for calculating first, second and third largest values
5. large[i] = largest(i,of values[*]) - largest value when i =1, second largest when i =2 and so on.
6. index = whichn(large[i],of values[*]) : Column index for kth largest values
7. names[i]    = vname(values[index]) :  Extract the variable name of the largest using largest index
8. drop i index; : Dropping Irrelevant Variables

1. The above program computes number of distinct values in variable 'y' when values of variable "x" is less than 30.
2. The keyword DISTINCT is used to remove or ignore duplicate records.
3. In the dataset, there are in total 3 cases in variable 'y' when x < 30. Whereas distinct number of cases in variable 'y' is equal to 2.

Output : Example 2

Example 3 : Case When

Example 4 : CASE WHEN with DISTINCT

data example;
input Srl x1-x5;
cards;
1 . 89 85 . 87
2 79 73 74 . .
3 80 95 . 95 .
;
run;

data out;
set example;
first = coalesce(of x1-x5);
last = coalesce(of x5-x1);
proc print noobs;
run;

1. First, create an array for variables from which we want to extract second non-missing value
2. The idea is to make first non-missing value missing so that we can pull second non-missing value
3. First, we need to create a temporary array of 5 variables in which we can store value.
4. The function WHICHN is used to find the column position of first non-missing value. For example, it returns 2 for row 1 values.
5. The DO LOOP is used to run the above steps iteratively in loop.

title1 "Weights of Students";
proc gchart data=temp;
hbar name / type=sum sumvar=weight;
run;

title1 "Weights of Students";
proc gchart data=temp;
vbar name / type=sum sumvar=weight;
run;

title1 "Weights of Students";
proc gchart data=temp;
vbar name / type=sum sumvar=weight outside=sum;
run;

title1 "Weights of Students";
proc gchart data=temp;
vbar name / type=sum sumvar=weight descending outside=sum;
run;

1. Assign Value Labels 'Male''Female' to X-axis. It is done via axis1 value = option.
2. Change the orientation of Y-axis title with 'a=90' in axis2 label.
3. Change the scale of Y axis with 'order=(0 to 150 by 25)'.
4. minor=none turns off minor tick marks.
5. offset=(0,0) remove space between the last number in the axis and the frame.

pattern1 v=solid color=red;
proc gchart data=temp;
vbar sex / width= 25 type=mean sumvar=weight descending
maxis=axis1 raxis=axis2 outside=mean coutline=gray;
run;
quit;

title1 "Average Male vs. Female Weights";
pattern1 value=solid color=pink;
pattern2 value=solid color=cx42C0FB;
proc gchart data=temp;
vbar sex / width= 25 type=mean sumvar=weight descending
maxis=axis1 raxis=axis2 outside=mean coutline=gray subgroup=sex;
run;
quit;

SAS Graphs : Multiple Colors

SAS : Percentage Chart

title1 "% of Male vs. Female Students";
goptions htext=13pt htitle=15pt;
axis1 label=none  value=(f="Arial/Bold""Female""Male"); ;
axis2 label=(a=90 f="Arial/Bold"'Percentage of Students') minor=none  offset=(0,0);
proc gchart data=temp;
vbar sex / width= 25 type=pct descending
maxis=axis1 raxis=axis2 inside= freq outside=pct;
run;
quit;

title;
axis1 label=('MSRP') minor=none offset=(0,0);
axis2 label=none;
axis3 label=none offset=(7,7);
proc gchart data=sashelp.cars;
vbar type  / discrete type=mean sumvar=msrp
group=origin coutline=gray raxis=axis1 maxis=axis2 gaxis=axis3 noframe;
run;
quit;

SAS : Grouped Bar Chart

title;
axis1 label=('MSRP') minor=none offset=(0,0);
axis2 label=none offset=(7,7);
proc gchart data=sashelp.cars;
vbar type / discrete type=mean sumvar=msrp
subgroup=origin coutline=gray width=10
raxis=axis1 maxis=axis2 noframe;
run;
quit;

SAS : Stacked Bar Chart

title;
goptions htext=10pt htitle=12pt;
proc gchart data=temp;
vbar weight / space=1 width=10 outside=freq levels=4 range;
run; quit;

Scatterplot with SAS

symbol1 value=circle height=3 interpol=none color=blue;
symbol2 value=circle height=3 interpol=none color=red;
proc gplot data=temp;
plot height*weight=sex;
run;

Box Plot

title "Height Distribution by Sex";
symbol1 value=circle height=3 interpol=none color=blue;
axis1 minor=none offset=(0,0);
axis2 offset=(20,20);
proc gplot data=temp;
plot height*sex=1 /
vaxis=axis1 haxis=axis2 noframe;
run;

title "Height Distribution by Sex";
symbol1 interpol=boxt bwidth=4 color=blue;
axis1 minor=none offset=(0,0);
axis2 offset=(20,20);
proc gplot data=temp;
plot height*sex=1 /
vaxis=axis1 haxis=axis2 noframe;
run;

title "Line Chart";
symbol1 value=none interpol=sm color=blue;
proc gplot data=sashelp.failure;
where cause ="Contamination" and Process = "Process A";
plot count*day;
run;
quit;

1. The first parameter of data.table i refers to rows. It implies subsetting rows. It is equivalent to WHERE clause in SQL
2. The second parameter of data.table j refers to columns. It implies subsetting columns (dropping / keeping). It is equivalent to SELECT clause in SQL.
3. The third parameter of data.table by refers to adding a group so that all calculations would be done within a group. Equivalent to SQL's GROUP BY clause.

mydata[, .(mean(arr_delay, na.rm = TRUE), mean(dep_delay, na.rm = TRUE)), by = origin]

setkey(mydata, NULL)
unique(mydata)

dt = mydata[, rank:=frank(-distance,ties.method = "min"), by=carrier]

dat = mydata[, cum:=cumsum(distance), by=carrier]

1. First, sort the CustomerID variable. We will have these values - {3,9,13,15,16,17,20}
2. Calculate the median (middle value) of the list i.e. 15.
3. We would check whether search_value = median? Is 17 =15? No. 17 > 15.
4. Ignore all the values that are less than or equal to 15 as 17 is a higher number than 15.
5. Now, we need to search in the remaining list i.e. 16, 17, 20. The middle value is 17.
6. Is 17 = 17? Yes. the value found.

1. PROC DATASETS
2. INDEX = Data Step Option
3. PROC SQL

proc datasets library=work nolist;
modify mydata;
index create custid;
quit;

data mydata (index=(names=(first last)));
set mydata;
run;

proc sql;
create index custid
on mydata;
quit;

data temp;
length demog $12.;
do k =1 to 20000000;
custid = k+1;
if mod(k,8)=0 then demog ='category i';
if mod(k,8)=1 then demog ='category ii';
if mod(k,8)=2 then demog ='category iii';
output;
end;
run;

data testing;
set temp;
where custid in (5467620,225,2671899, 18000000);
run;

proc datasets library=work;

modify temp;

index create custid;

quit;

data testing2;

set temp;

where custid in (5467620,225,2671899, 18000000);

run;

data a_index (index=(id));
set a;
run;
data b_index (index=(id));
    set b;
 run;

data final;
merge a_index(in=a) b_index(in=b);
by id;
if a=b;
run;

If you are merging/joining a small table with the large indexed table, the indexing would result a good performance. By 'small' table, it means at most 15% of the large indexed table. If you are merging two large indexed table, it might reduce the performance as it takes resources to create an index.

• PROC DATASETS : Use command 'index delete index-name;'
• PROC SQL : Use command 'drop index index-name from dataset-name;'

1. Type the text 'No. of Non-Missing Values' in cell B3 and enter any dummy value (let's say 117) corresponding to it in cell C3. We'll calculate the exact value in the following steps.
2. Enter the text 'No. of Missing Values'in cell B4and the actual figure (let's say 50) in cell C4.
3. Type '% of Non-Missing Values'in cell B5 and enter the following formula in cell C5.

C3/(C3+C4)

Goal Seek Menu

4. Go to the 'Data' tab and click on 'What-If Analysis' button and select Goal Seek under it. See and follow the instructions shown in the image above.
5. When you click on Goal Seek button, it will open a box in which you need to provide cell reference and target value. The explanation is provided below -

Goal Seek Parameters

1. Enter a formula =150-C3 in cell C4 (As shown in the image above)
2. Run Goal Seek Analysis again (Follow the instructions below)

Goal Seek : Instructions

1. The cell reference in Set cell: box must contain a formula. Otherwise, it would return an error.
2. You cannot give cell reference in 'To value:' box. You need to enter a value.
3. The cell reference in 'By changing cell:' cannot contain a formula. It must be a typed value.

1. Scandinavian Airlines (SAS Flights)
2. Sequences Annotated by Structure (SAS Structural and Molecular Biology)
3. Serial Attached SCSI technology
4. Singapore American School
5. Society for Amateur Scientists
6. Society for Applied Spectroscopy
7. Southern Astronomical Society
8. Special Air Service (SAS Force)
9. Surfers Against Sewage

data temp;
x = 1234567890;
x1 = 1234567890123456;
put x= x1=;
run;

SAS : Length of Numeric Variable

1.23456789E15 is equivalent to 1.23456789 𝗑 10¹⁵

data temp;
x = 1234567890;
x1 = 1234567890123456;
format x1 best16.;
put x= x1=;
run;